Update config.py #150
Update config.py #150
Conversation
fix(embeddings): increase chunk_size for OpenAIEmbeddings to avoid API limits Adjusted the `chunk_size` parameter in the `OpenAIEmbeddings` class to 200, based on the calculation: - Maximum payload size: 300 000 tokens - Upper‐bound chunk size: 1 500 tokens - ⇒ 300 000 / 1 500 = 200 This ensures embedding requests stay within OpenAI’s maximum input size and prevents failures due to oversized requests.
|
Were you not able to customize the chunk size via the .env variable? See the Readme.md |
|
The two settings are not the same. The For example, if you set chunk_size to 1000 in the .env file and load a document that produces more than 300 chunks, the OpenAI API will return an error because it supports a maximum of 300,000 tokens per call. |
| @@ -190,6 +190,7 @@ def init_embeddings(provider, model): | |||
| api_key=RAG_OPENAI_API_KEY, | |||
| openai_api_base=RAG_OPENAI_BASEURL, | |||
| openai_proxy=RAG_OPENAI_PROXY, | |||
| chunk_size=200 | |||
There was a problem hiding this comment.
rather than hardcode a value, it would be better to make this configurable
|
I have the same problem, but with Azure OpenAI. I have opened a separate pull request that implements the requested change and applies it to both OpenAI and Azure OpenAI: |
fix(embeddings): increase chunk_size for OpenAIEmbeddings to avoid API limits
Adjusted the
chunk_sizeparameter in theOpenAIEmbeddingsclass to 200, based on the calculation:This ensures embedding requests stay within OpenAI’s maximum input size and prevents failures due to oversized requests.