Enhancement: Add Amazon Nova models to Bedrock defaults

[RenFraser]

What features would you like to see added?

We should add the Amazon Nova models to the supported list of Bedrock models.

The list of model ids can be found in the AWS docs here. I'd like to add are as follows:

amazon.nova-canvas-v1:0
amazon.nova-lite-v1:0	
amazon.nova-micro-v1:0	
amazon.nova-pro-v1:0	
amazon.nova-reel-v1:0	

These models don't currently appear in the default list of models when commenting out the BEDROCK_AWS_MODELS in .env and loading the Bedrock provider. You can access the models by setting them directly in the env variable so think they should appear within the default list when unset.

More details

It looks like we just need to update this line to include the aforementioned model ids.

Which components are impacted by your request?

General

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[RenFraser]

I'm happy to submit a PR for this.

[dvejsada]

Just be aware that Canvas and Reel are image/video generation models, not text models.

[RenFraser]

Thanks. Perhaps we should stick to the text models in the first PR, then.

[dvejsada]

Thanks. Perhaps we should stick to the text models in the first PR, then.

Yes, the image/video generation models will not work directly in LibreChat as endpoint, but rather need to be used via plugin or MCP tool (see this as example).

[RenFraser]

Thank you @dvejsada, that's helpful to know! :)

[danny-avila]

Hi @RenFraser

I won't be adding them to the list because they may more than likely require region identifiers and don't want to assume the region for the default list, i.e.

Using amazon.nova-lite-v1:0

2025-02-06T21:30:21.020Z error: [handleAbortError] AI response error; aborting request: Invocation of model ID amazon.nova-lite-v1:0 with on-demand throughput isn’t supported

but us.amazon.nova-lite-v1:0 works for me.

[RenFraser]

@danny-avila That's strange. I'm not required to use cross-region inference endpoints. Is model access enabled in another US region but not in your current/default region?

Here's an example where it works for me without cross-region inference. If they do end up requiring cross-region inference endpoints, should we defer the removal of them until that becomes the case?

.env

BEDROCK_AWS_DEFAULT_REGION=us-east-1
BEDROCK_AWS_MODELS=amazon.nova-lite-v1:0

Screenshot

As a side thought, I wonder if it would be a nice feature to call ListFoundationModels to populate the default list? Then it would only show models that the user has access to and can use.

[danny-avila]

That's strange. I'm not required to use cross-region inference endpoints. Is model access enabled in another US region but not in your current/default region?

I'm not sure, but this has been the requirement for me when using newer models, starting from the latest 3.5-sonnet models, then llama 3.3, now nova, etc.

As a side thought, I wonder if it would be a nice feature to call ListFoundationModels to populate the default list? Then it would only show models that the user has access to and can use.

I agree this could be a better solution. Leaving this here as a note for myself

https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/bedrock/command/ListFoundationModelsCommand/

[RenFraser]

Okay, that seems like a better longer term solution. Happy to cut an issue if you'd like?