This is a Bedrock (and other generative AI tools from AWS) API forwarding tool that can issue virtual keys, log chats, and manage costs.
It is compatible with any OPENAI client that can define Host and API Key.
Docker Image: DockerHub, Public ECR
Docker Image for Lambda: DockerHub, Public ECR
- Supports all current and future large language models from Bedrock (supported through bedrock-converse).
- Supports models deployed through Sagemaker LMI (partial models).
- Supports other forms of custom models, including Ollama, etc.
- More AI workflow applications, such as internet search, AWS command executors, etc.
- Create API keys. Keys can be created for regular users and administrators. Regular users can chat, while administrators can manage API keys and costs.
- Record the cost of each call and use it as a basis for cost control.
- Cost control. You can set monthly quotas and account balances for each API key. When the monthly quota or account balance is insufficient, it cannot be used.
- Calculate the overall cost.
Important
The cost calculation of this project cannot be used as the basis for AWS billing. Please refer to the AWS bill for actual expenses. Please refer to the official website for the Bedrock pricing.
Models and their parameters can be defined from the backend.
Once defined, models can be bound to groups or API Keys.
- Added caching mechanism, with the following details:
-
If the system does not configure environment variables PGSQL_HOST or PGSQL_DATABASE, BRConnector will switch to pass-through mode. In this mode, only the ADMIN_API_KEY environment variable can be used to access models, and only built-in static models can be accessed (default LLM models supported by Bedrock Converse; if new models are released in the future, BRConnector code needs to be updated).
-
If a database is configured and the PERFORMANCE_MODE environment variable is set, custom models and API Keys configured in the database will take effect. These two parts of data will be loaded directly into memory (up to 2000 entries, refreshed every 1 minute, significantly reducing database access). In this case, BRConnector will only verify if the API Key is valid, without checking if the API Key has permission to access a specific model. It will no longer save conversation records or track and control costs.
-
Fixed a Dify access bug. Issue #43
-
For Feishu webhook, added multi-turn dialogue, image recognition, and document recognition features.
-
Added optional configurations for NoDB, PERFORMANCE_MODE in CloudFormation.
-
Upgraded the aws-lambda-adapter image for Lambda to version 0.8.4. The new adapter version resolves conflicts with the Authorization Header in IAM_Auth for Lambda function URLs.
-
Optimized CloudFormation Output, now showing clear endpoints.
-
New Feature: Introduced a webhook mechanism for easy integration with third-party applications.
-
New Webhook: Feishu, allowing BRConnector integration into Feishu through configuration.
-
New Plugin: continue-coder, enabling BRConnector configuration in Continue for code generation.
-
New Models Support: The painter plugin now supports the latest bedrock SD models:
stability.stable-image-core-v1:0,stability.sd3-large-v1:0, andstability.stable-image-ultra-v1:0. -
Deployment Script Upgrade: Added a permission configuration, now correctly calling cross-region model profiles.
-
Improvement: Added an API key query API, allowing precise matching by name, group_id, and role.
-
Improvement: Default model support: Now a model named 'default' can respond to any model name. If not defined, an error message will be output.
-
Refactored parts of the code and added a standard API:
/v1/completions, to support continue-coder. Currently, only bedrock-coverse and continue-coder have implemented this API. -
Upgraded the underlying AWS SDK dependency to 3.645.0 and fixed various other bugs.
This connector is an open-source software aimed at providing proxy services for using Bedrock services. We make our best efforts to ensure the security and legality of the software, but we are not responsible for the users' behavior.
The connector is intended solely for personal learning and research purposes. Users shall not use it for any illegal activities, including but not limited to hacking, spreading illegal information, etc. Otherwise, users shall bear the corresponding legal responsibilities themselves. Users are responsible for complying with the laws and regulations in their respective jurisdictions and shall not use the connector for any illegal or non-compliant purposes. The developers and maintainers of this software shall not be liable for any disputes, losses, or legal liabilities arising from the use of this connector.
We reserve the right to modify or terminate the connector's code at any time without further notice. Users are expected to understand and comply with the relevant local laws and regulations.
If you have any questions regarding this disclaimer, please feel free to contact us through the open-source channels.
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.