Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added path param tab http #1813

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

feat: added path param tab http #1813

wants to merge 2 commits into from

Conversation

aashishmalik
Copy link
Contributor

@aashishmalik aashishmalik commented Nov 20, 2024

What are the changes introduced in this PR?

Write a brief explainer on your code changes.

What is the related Linear task?

Resolves INT-XXX

Please explain the objectives of your changes below

Put down any required details on the broader aspect of your changes. If there are any dependent changes, mandatorily mention them here

Any changes to existing capabilities/behaviour, mention the reason & what are the changes ?

N/A

Any new dependencies introduced with this change?

N/A

Any new checks got introduced or modified in test suites. Please explain the changes.

N/A


Developer checklist

  • My code follows the style guidelines of this project

  • No breaking changes are being introduced.

  • All related docs linked with the PR?

  • All changes manually tested?

  • Any documentation changes needed with this change?

  • I have executed schemaGenerator tests and updated schema if needed

  • Are sensitive fields marked as secret in definition config?

  • My test cases and placeholders use only masked/sample values for sensitive fields

  • Is the PR limited to 10 file changes & one task?

Reviewer checklist

  • Is the type of change in the PR title appropriate as per the changes?

  • Verified that there are no credentials or confidential data exposed with the changes.

Summary by CodeRabbit

  • New Features

    • Introduced a new property for specifying path parameters in HTTP requests.
    • Added a section in the user interface for entering path parameters in sequence, enhancing configuration options.
    • Updated JSON schema to allow additional properties, increasing flexibility in configuration.
  • Bug Fixes

    • No bug fixes were included in this release.
  • Documentation

    • Updated configuration documentation to reflect new properties and options available for HTTP destinations.

Copy link

coderabbitai bot commented Nov 20, 2024

Walkthrough

The pull request introduces changes to the HTTP configuration files in the project. A new property, "pathParams", is added to the defaultConfig array in db-config.json, enhancing the configuration options for HTTP destinations. Additionally, the JSON schema in schema.json is updated to allow for additionalProperties, enabling flexibility in the configuration object. Lastly, a new section for "Path Parameters" is introduced in ui-config.json, providing a user interface for entering path parameters in HTTP requests.

Changes

File Path Change Summary
src/configurations/destinations/http/db-config.json Added property "pathParams" in defaultConfig array within destConfig object.
src/configurations/destinations/http/schema.json Added "additionalProperties": true in configSchema to allow unspecified properties.
src/configurations/destinations/http/ui-config.json Introduced a new section for "Path Parameters" in redirectGroups with a mapping field.

Possibly related PRs

  • fix: http destination fixes #1740: This PR modifies the db-config.json file, which is directly related to the same file modified in the main PR, although it does not introduce the "pathParams" property.
  • fix: http enchancements #1759: This PR also modifies the ui-config.json file, which is relevant as it deals with the same configuration file where the main PR introduced a new section for path parameters.

Suggested reviewers

  • lvrach
  • am6010
  • AchuthaSourabhC
  • krishna2020
  • ssbeefeater
  • debanjan97
  • cisse21
  • ruchiramoitra
  • 1abhishekpandey
  • ItsSudip
  • shrouti1507
  • utsabc
  • sandeepdsvs

🐇 In the meadow, changes bloom,
New paths for requests, making room.
With parameters now in sight,
Configs grow, oh what a delight!
JSON sings, UI shines bright,
Hop along, let’s code tonight! 🌟


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@aashishmalik
Copy link
Contributor Author

Screenshot 2024-11-20 at 4 00 10 AM

Copy link

codecov bot commented Nov 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (5f58915) to head (76df956).

Additional details and impacted files
@@            Coverage Diff            @@
##           develop     #1813   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            2         2           
  Lines           53        53           
  Branches         7         7           
=========================================
  Hits            53        53           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (2)
src/configurations/destinations/http/db-config.json (1)

48-48: Consider documenting path parameter security guidelines.

While path parameters are added as a configuration option, it's important to ensure they are properly validated to prevent path traversal attacks.

Consider:

  1. Adding validation rules in the schema to restrict path parameter formats
  2. Documenting best practices for path parameter usage in the integration guide
  3. Implementing server-side sanitization of path parameters
src/configurations/destinations/http/ui-config.json (1)

600-618: Add validation and improve documentation for path parameters

The implementation follows the existing pattern well, but could benefit from these improvements:

  1. Add JSONPath validation like other mapping fields:
 {
   "type": "mapping",
   "label": "Enter your path parameters in sequence",
+  "jsonPathValidation": true,
   "columns": [
  1. Add a note explaining the relationship with API URL:
 {
   "type": "mapping",
   "label": "Enter your path parameters in sequence",
+  "note": [
+    "Path parameters will be applied in sequence to replace variables in your API URL. For example, if your URL is 'http://api.example.com/users/{{param1}}/posts/{{param2}}', the first path parameter will replace {{param1}} and the second will replace {{param2}}.",
+    {
+      "text": "Learn more",
+      "link": "https://www.rudderstack.com/docs/destinations/http/setup-guide/#path-parameters"
+    }
+  ],
   "columns": [
  1. Consider adding an order column to make the sequence more explicit:
   "columns": [
+    {
+      "key": "order",
+      "type": "textInput",
+      "label": "Order",
+      "placeholder": "1",
+      "disabled": true
+    },
     {
       "key": "path",
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 5f58915 and 76df956.

📒 Files selected for processing (3)
  • src/configurations/destinations/http/db-config.json (1 hunks)
  • src/configurations/destinations/http/schema.json (1 hunks)
  • src/configurations/destinations/http/ui-config.json (1 hunks)
🔇 Additional comments (1)
src/configurations/destinations/http/schema.json (1)

6-6: LGTM! Consider documenting the schema change.

The addition of "additionalProperties": true is necessary to support the new path parameters feature. While this relaxes the schema validation to allow undefined properties, it maintains the required property validations and existing constraints.

Let's verify the integration with related changes:

✅ Verification successful

Schema change is properly integrated across configurations

The verification confirms that pathParams is properly integrated:

  • Present in UI config as a configurable field with appropriate label and placeholder
  • Listed in DB config as one of the supported configuration properties
  • Schema change with additionalProperties: true enables this new property
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the integration of schema changes with UI and DB configs
# Expected: Find corresponding path parameter configurations in UI and DB config files

# Check for path parameter configurations in UI config
echo "Checking UI config for path parameter settings..."
rg -l "pathParams" "src/configurations/destinations/http/ui-config.json"

# Check for path parameter configurations in DB config
echo "Checking DB config for path parameter settings..."
rg -l "pathParams" "src/configurations/destinations/http/db-config.json"

Length of output: 463


Script:

#!/bin/bash
# Let's examine the actual implementation of pathParams in both files
echo "UI Config pathParams implementation:"
rg -A 5 -B 5 "pathParams" "src/configurations/destinations/http/ui-config.json"

echo -e "\nDB Config pathParams implementation:"
rg -A 5 -B 5 "pathParams" "src/configurations/destinations/http/db-config.json"

Length of output: 853

@@ -45,6 +45,7 @@
"propertiesMapping",
"queryParams",
"headers",
"pathParams",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Schema validation missing for pathParams

The schema.json file lacks validation rules for the pathParams configuration, while the UI configuration (ui-config.json) correctly includes the Path Parameters tab. This could lead to validation issues.

  • Add schema validation for pathParams in schema.json similar to the existing queryParams and headers validation:
"pathParams": {
  "type": "array",
  "items": {
    "type": "object",
    "properties": {
      "path": {
        "type": "string",
        "pattern": "(^\\{\\{.*\\|\\|(.*)\\}\\}$)|(^env[.].+)|^(.{0,100})$"
      }
    }
  }
}
🔗 Analysis chain

LGTM! The placement of pathParams is logical and non-breaking.

The addition of "pathParams" is well-positioned among other HTTP request configuration options like queryParams and headers.

Let's verify the integration with related schema and UI files:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that schema.json and ui-config.json have corresponding changes for pathParams

# Check if schema.json has been updated to allow pathParams
echo "Checking schema.json for pathParams support..."
rg -A 5 '"pathParams"' "src/configurations/destinations/http/schema.json"

# Check if ui-config.json has UI configuration for pathParams
echo "Checking ui-config.json for pathParams UI configuration..."
rg -A 5 '"pathParams"' "src/configurations/destinations/http/ui-config.json"

Length of output: 482


Script:

#!/bin/bash
# Let's verify schema.json changes more thoroughly by checking its content
echo "Checking complete schema.json content..."
cat "src/configurations/destinations/http/schema.json"

# Also check if there are any validation rules or type definitions for pathParams
echo -e "\nChecking for pathParams validation rules..."
rg -C 10 'pathParams|path.*param' "src/configurations/destinations/http/schema.json"

# Check the complete UI config to understand the context better
echo -e "\nChecking complete ui-config.json content..."
cat "src/configurations/destinations/http/ui-config.json"

Length of output: 51198

Comment on lines 600 to 618
},
{
"name": "Path Parameters",
"fields": [
{
"type": "mapping",
"label": "Enter your path parameters in sequence",
"columns": [
{
"key": "path",
"type": "textInput",
"label": "Path",
"placeholder": "$.userId"
}
],
"default": [],
"configKey": "pathParams"
}
]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Documentation needs update to reference path parameters feature

The API URL field's documentation currently mentions dynamic path variables using JSON template syntax (e.g., {{$.traits.userId}}), but doesn't reference the new dedicated path parameters feature. The documentation should be updated to:

  • Mention the new Path Parameters section
  • Clarify the relationship between URL templating and path parameters
  • Guide users on when to use each approach
🔗 Analysis chain

Verify documentation for path parameter usage

The new path parameters feature needs to be properly referenced in the API URL field's documentation.

Let's check if the API URL field's documentation mentions path parameters:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for API URL field documentation in the file
rg -U "API URL.*note.*path" src/configurations/destinations/http/ui-config.json

Length of output: 81


Script:

#!/bin/bash
# Let's search for the API URL field configuration to check its documentation
rg -U "API URL" -A 10 src/configurations/destinations/http/ui-config.json

Length of output: 1300

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 76df956 and 26928a1.

📒 Files selected for processing (1)
  • src/configurations/destinations/http/ui-config.json (1 hunks)
🧰 Additional context used
🪛 Biome (1.9.4)
src/configurations/destinations/http/ui-config.json

[error] 616-616: Expected a property but instead found '}'.

Expected a property here.

(parse)

🔇 Additional comments (1)
src/configurations/destinations/http/ui-config.json (1)

600-601: Update API URL field documentation to reference path parameters

The API URL field's documentation needs to be updated to mention the new Path Parameters section and clarify when to use path parameters versus template syntax.

Comment on lines +614 to +615
"regexErrorMessage": "Please enter a valid JSON path or a constant",
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix JSON syntax error: Remove trailing comma

There's a trailing comma after the regexErrorMessage field which needs to be removed to ensure valid JSON syntax.

Apply this fix:

           "regex": "^\\$(\\.(\\w+|\\*)|\\[\\d+\\]|\\[('\\w+'|\"\\w+\")\\]|\\[\\*\\]|\\.\\w+\\(\\))*$|^(?!\\$).{1,100}$",
-          "regexErrorMessage": "Please enter a valid JSON path or a constant",
+          "regexErrorMessage": "Please enter a valid JSON path or a constant"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"regexErrorMessage": "Please enter a valid JSON path or a constant",
}
"regex": "^\\$(\\.(\\w+|\\*)|\\[\\d+\\]|\\[('\\w+'|\"\\w+\")\\]|\\[\\*\\]|\\.\\w+\\(\\))*$|^(?!\\$).{1,100}$",
"regexErrorMessage": "Please enter a valid JSON path or a constant"
}

Comment on lines +600 to +620
},
{
"name": "Path Parameters",
"fields": [
{
"type": "mapping",
"label": "Enter your path parameters in sequence",
"columns": [
{
"key": "path",
"type": "textInput",
"label": "Path",
"placeholder": "$.userId or users",
"regex": "^\\$(\\.(\\w+|\\*)|\\[\\d+\\]|\\[('\\w+'|\"\\w+\")\\]|\\[\\*\\]|\\.\\w+\\(\\))*$|^(?!\\$).{1,100}$",
"regexErrorMessage": "Please enter a valid JSON path or a constant",
}
],
"default": [],
"configKey": "pathParams"
}
]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Documentation needs to be added for path parameters feature

The codebase search reveals:

  • Path parameters UI configuration exists but lacks comprehensive documentation
  • Example URLs with path parameters are only present in test files
  • The API URL field's documentation briefly mentions dynamic path variables but doesn't explain the new path parameters feature

Required documentation updates:

  • Add path parameters documentation in README.md or create a new documentation file
  • Include examples demonstrating the relationship between path parameters and API URL
  • Document the syntax and usage of the new path parameters feature
🔗 Analysis chain

Verify path parameter documentation and examples

Let's ensure the documentation and examples are consistent across the codebase.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for path parameter documentation and examples
echo "Searching for path parameter documentation..."
rg -l "path param(eter)?" docs/
rg -l "path param(eter)?" README.md

echo "Searching for example URLs with path parameters..."
rg -l "http.*\{\{.*\}\}" docs/

Length of output: 408


Script:

#!/bin/bash
# Let's try a broader search for documentation and examples
echo "Searching for path parameter documentation in all files..."
rg -i "path param(eter)?"

echo -e "\nSearching for example URLs with path parameters..."
rg -i "https?://[^\"'\s]*\{[^}]+\}"

echo -e "\nSearching for any markdown or documentation files..."
fd -e md
fd -e mdx
fd -e txt

Length of output: 2693

🧰 Tools
🪛 Biome (1.9.4)

[error] 616-616: Expected a property but instead found '}'.

Expected a property here.

(parse)

Comment on lines +602 to +620
"name": "Path Parameters",
"fields": [
{
"type": "mapping",
"label": "Enter your path parameters in sequence",
"columns": [
{
"key": "path",
"type": "textInput",
"label": "Path",
"placeholder": "$.userId or users",
"regex": "^\\$(\\.(\\w+|\\*)|\\[\\d+\\]|\\[('\\w+'|\"\\w+\")\\]|\\[\\*\\]|\\.\\w+\\(\\))*$|^(?!\\$).{1,100}$",
"regexErrorMessage": "Please enter a valid JSON path or a constant",
}
],
"default": [],
"configKey": "pathParams"
}
]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider enhancing path parameter validation and documentation

The implementation needs additional safeguards and clarity:

  1. Add validation to prevent duplicate path parameters which could cause runtime issues
  2. Clarify the sequence importance in the label or add a note
  3. Consider adding validation for the total number of path parameters to match URL placeholders

Apply these changes to enhance validation and clarity:

 {
   "name": "Path Parameters",
   "fields": [
     {
       "type": "mapping",
-      "label": "Enter your path parameters in sequence",
+      "label": "Enter your path parameters in sequence (order matters)",
+      "note": "Parameters will be substituted in order into URL placeholders. Use either JSON path (e.g., $.userId) for dynamic values or constants.",
       "columns": [
         {
           "key": "path",
           "type": "textInput",
           "label": "Path",
           "placeholder": "$.userId or users",
           "regex": "^\\$(\\.(\\w+|\\*)|\\[\\d+\\]|\\[('\\w+'|\"\\w+\")\\]|\\[\\*\\]|\\.\\w+\\(\\))*$|^(?!\\$).{1,100}$",
           "regexErrorMessage": "Please enter a valid JSON path or a constant",
+          "validation": {
+            "unique": true,
+            "errorMessage": "Duplicate path parameters are not allowed"
+          }
         }
       ],
       "default": [],
       "configKey": "pathParams"
     }
   ]
 }

Committable suggestion skipped: line range outside the PR's diff.

🧰 Tools
🪛 Biome (1.9.4)

[error] 616-616: Expected a property but instead found '}'.

Expected a property here.

(parse)

@devops-github-rudderstack
Copy link
Contributor

This PR is considered to be stale. It has been open for 20 days with no further activity thus it is going to be closed in 7 days. To avoid such a case please consider removing the stale label manually or add a comment to the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants