stop parameter fixed #1805
stop parameter fixed #1805
Conversation
WalkthroughThe changes update the Dockerfile and metadata configuration. In the Dockerfile, the Python environment variable is adjusted from Changes
Suggested reviewers
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
docker/Dockerfile(1 hunks)metadata/llm_metadata.yml(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (3)
- GitHub Check: Codacy Static Code Analysis
- GitHub Check: Analyze (python)
- GitHub Check: Python CI
🔇 Additional comments (3)
docker/Dockerfile (2)
10-10: LGTM! Development packages added.The additional development packages (
ncurses-devel,readline-devel,xz-devel,tk-devel,bzip2,patch,sqlite) are appropriate for Python development environment.
3-3: Consider being more specific with Python version.While using
3.10works, consider using a specific patch version (e.g.,3.10.16) to ensure consistent builds across environments.❌ Verification inconclusive
🏁 Script executed:
#!/bin/bash # Check available Python 3.10.x versions pyenv install --list | grep "^ 3.10"Length of output: 135
Clarification on Python Version Specification
While using
PYTHON_VERSION=3.10may offer flexibility, specifying a complete patch version (for example,3.10.16) helps ensure consistent builds across environments. The initial verification script failed because thepyenvcommand wasn’t found in the current environment. Please verify manually in your build context that a specific patch version is available (or confirm that using the flexible version specifier does not lead to unexpected behavior).metadata/llm_metadata.yml (1)
273-279: Verify the stop parameter implementation.The new implementation of the
stopparameter is more restrictive, only allowing array type. Please confirm this is the intended behavior for AWS Nova models.✅ Verification successful
🌐 Web query:
What is the correct type for the stop parameter in AWS Bedrock Nova models?💡 Result:
The
stopparameter (referred to asstopSequencesin configuration) for AWS Bedrock Nova models must be specified as an array/list of strings. Each string represents a character sequence that triggers the model to halt further token generation when encountered[1][7][8].Key details:
- Type: Array/List (
[]syntax)- Element type: Strings (e.g.,
["\n\n", "END"])- Maximum entries: 4 sequences[8]
- Minimum length: Each sequence must be ≥1 character[8]
Example configuration from search results:
config: stopSequences: ['END'] # Nova model example[2]This matches Amazon's API specification where
stopSequencesappears in request schemas as:{ "stopSequences": ["string"] # Array of strings[7][8] }Citations:
- 1: https://docs.aws.amazon.com/bedrock/latest/userguide/inference-parameters.html?icmpid=docs_bedrock_help_panel_playgrounds
- 2: https://www.promptfoo.dev/docs/providers/aws-bedrock/
- 3: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-cohere-command.html
- 4: https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-stop.html
- 5: https://hidekazu-konishi.com/entry/amazon_bedrock_for_titling_commenting_ocr_with_amazon_nova_pro.html
- 6: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-text.html?trk=28cb67ec-6e8f-4131-96c3-3070118f5815&sc_channel=el
- 7: https://docs.aws.amazon.com/pdfs/nova/latest/userguide/nova-ug.pdf
- 8: https://docs.aws.amazon.com/pt_br/bedrock/latest/APIReference/API_runtime_InferenceConfiguration.html
- 9: https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-guidelines.html
- 10: https://docs.aws.amazon.com/nova/latest/userguide/tool-use.html
Stop Parameter Implementation Confirmed
The
stopparameter is correctly defined as an array of strings (with a maximum of 4 items), which is in line with the expected configuration for AWS Bedrock Nova models.🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 279-279: no new line character at the end of file
(new-line-at-end-of-file)
| aws-nova: | ||
| $schema: "https://json-schema.org/draft/2020-12/schema" | ||
| type: "object" | ||
| description: "AWS Nova AI Models for Prompt" | ||
| properties: | ||
| temperature: | ||
| type: "number" | ||
| default: 0.0 | ||
| minimum: 0.0 | ||
| maximum: 2.0 | ||
| description: "The temperature hyperparameter controls the creativity or randomness of the generated responses." | ||
| max_tokens: | ||
| type: "integer" | ||
| default: 300 | ||
| minimum: 5 | ||
| maximum: 5120 | ||
| description: "The max_tokens hyperparameter limits the length of generated responses in chat completion." | ||
| model: | ||
| type: "string" | ||
| default: "bedrock/converse/us.amazon.nova-lite-v1:0" | ||
| enum: ["bedrock/converse/us.amazon.nova-micro-v1:0", "bedrock/converse/us.amazon.nova-lite-v1:0", "bedrock/converse/us.amazon.nova-pro-v1:0"] | ||
| description: "The model hyperparameter is the ID of the model to use such as gpt-2, gpt-3, or a custom model that you have trained or fine-tuned." | ||
| top_p: | ||
| type: "number" | ||
| default: 0.0 | ||
| minimum: 0.0 | ||
| maximum: 1.0 | ||
| description: "The top_p hyperparameter is a value that controls the diversity of the generated responses." | ||
| stop: | ||
| anyOf: | ||
| - type: "array" | ||
| maxItems: 4 | ||
| items: | ||
| type: "string" | ||
| - type: "null" | ||
| type: | ||
| - "array" | ||
| - "null" | ||
| default: null | ||
| description: "The stop hyperparameter is used to specify a list of tokens that should be used to indicate the end of a generated response." |
There was a problem hiding this comment.
Remove duplicate aws-nova section.
The file contains two identical aws-nova sections with different implementations of the stop parameter. Based on the PR title "stop parameter fixed", it appears the second implementation (array-only) is intended, but the old section wasn't removed.
Remove the first duplicate section:
-aws-nova:
- $schema: "https://json-schema.org/draft/2020-12/schema"
- type: "object"
- description: "AWS Nova AI Models for Prompt"
- properties:
- temperature:
- type: "number"
- default: 0.0
- minimum: 0.0
- maximum: 2.0
- description: "The temperature hyperparameter controls the creativity or randomness of the generated responses."
- max_tokens:
- type: "integer"
- default: 300
- minimum: 5
- maximum: 5120
- description: "The max_tokens hyperparameter limits the length of generated responses in chat completion."
- model:
- type: "string"
- default: "bedrock/converse/us.amazon.nova-lite-v1:0"
- enum: ["bedrock/converse/us.amazon.nova-micro-v1:0", "bedrock/converse/us.amazon.nova-lite-v1:0", "bedrock/converse/us.amazon.nova-pro-v1:0"]
- description: "The model hyperparameter is the ID of the model to use such as gpt-2, gpt-3, or a custom model that you have trained or fine-tuned."
- top_p:
- type: "number"
- default: 0.0
- minimum: 0.0
- maximum: 1.0
- description: "The top_p hyperparameter is a value that controls the diversity of the generated responses."
- stop:
- anyOf:
- - type: "array"
- maxItems: 4
- items:
- type: "string"
- - type: "null"
- type:
- - "array"
- - "null"
- default: null
- description: "The stop hyperparameter is used to specify a list of tokens that should be used to indicate the end of a generated response."Also applies to: 245-279
Summary by CodeRabbit