Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use AWS_DEFAULT_REGION for bedrock if provided #157

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,19 +40,19 @@ pip install -r requirements.txt
AG-A supports using both AWS Bedrock and OpenAI as LLM model providers. You will need to set up API keys for the respective provider you choose. By default, AG-A uses AWS Bedrock for its language models.

#### AWS Bedrock Setup
AG-A integrates with AWS Bedrock by default. To use AWS Bedrock, you will need to configure your AWS credentials and region settings:
To use AWS Bedrock, you will need to configure your AWS credentials and region settings:

```bash
export AWS_DEFAULT_REGION="<your-region>"
export AWS_ACCESS_KEY_ID="<your-access-key>"
export AWS_SECRET_ACCESS_KEY="<your-secret-key>"
```

Ensure you have an active AWS account and appropriate permissions set up for using Bedrock models. You can manage your AWS credentials through the AWS Management Console. See [Bedrock supported AWS regions](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-regions.html)
Ensure you have an active AWS account and appropriate permissions set up for using Bedrock models. You can manage your AWS credentials through the AWS Management Console. See [Bedrock supported AWS regions](https://docs.aws.amazon.com/bedrock/latest/userguide/models-regions.html).


#### OpenAI Setup
To use OpenAI, you'll need to set your OpenAI API key as an environment variable:
To use OpenAI, you will need to set your OpenAI API key as an environment variable:

```bash
export OPENAI_API_KEY="sk-..."
Expand Down
13 changes: 6 additions & 7 deletions src/autogluon/assistant/llm/llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,7 @@ def get_openai_models() -> List[str]:
@staticmethod
def get_bedrock_models() -> List[str]:
try:
# TODO: Remove hardcoding AWS region
bedrock = boto3.client("bedrock", region_name="us-west-2")
bedrock = boto3.client("bedrock")
response = bedrock.list_foundation_models()
return [model["modelId"] for model in response["modelSummaries"]]
except Exception as e:
Expand Down Expand Up @@ -166,8 +165,6 @@ def _get_bedrock_chat_model(config: DictConfig) -> AssistantChatBedrock:
"temperature": config.temperature,
"max_tokens": config.max_tokens,
},
# TODO: Remove hardcoding AWS region
region_name="us-west-2",
verbose=config.verbose,
)

Expand All @@ -177,9 +174,11 @@ def get_chat_model(cls, config: DictConfig) -> Union[AssistantChatOpenAI, Assist
assert config.provider in valid_providers, f"{config.provider} is not a valid provider in: {valid_providers}"

valid_models = cls.get_valid_models(config.provider)
assert (
config.model in valid_models
), f"{config.model} is not a valid model in: {valid_models} for provider {config.provider}"
assert config.model in valid_models, (
f"{config.model} is not a valid model in: {valid_models} for provider {config.provider}. "
"Please check if the requested model is available in "
f"`AWS_DEFAULT_REGION={os.environ.get('AWS_DEFAULT_REGION')}`."
)

if config.model not in WHITE_LIST_LLM:
logger.warning(f"{config.model} is not on the white list. Our white list models include {WHITE_LIST_LLM}")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,7 @@ def __init__(
num_iterations: int = 2,
optimization_metric: str = "roc",
eval_model: str = "lightgbm",
# TODO: Remove hardcoding AWS region
region_name: str = "us-west-2",
region_name: str = os.environ.get("AWS_DEFAULT_REGION", "us-west-2"),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tonyhoo it is still required to pass the region_name here as for CAAFE, so I think it is okay to keep this.

Later after the release I'm going to refactor CAAFE, and then this argument can also be removed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to keep the region setting consistent across the repo

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not possible to remove it from CAAFE until we make changes in CAAFE upstream. Would you suggest that I also make changes upstream? See details here:

https://github.com/AnirudhDagar/CAAFE/blob/be58d15e03329e0a7344938d347e07b76166523f/caafe/caafe.py#L37

https://github.com/AnirudhDagar/CAAFE/blob/be58d15e03329e0a7344938d347e07b76166523f/caafe/sklearn_wrapper.py#L42

**kwargs,
) -> None:
# Set up credentials if using OpenAI
Expand Down
Loading