Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The chunk size of the knowledge base cannot be modified through environment variables. #12500

Open
5 tasks done
Moiyu1998 opened this issue Jan 8, 2025 · 4 comments · May be fixed by #12611
Open
5 tasks done

The chunk size of the knowledge base cannot be modified through environment variables. #12500

Moiyu1998 opened this issue Jan 8, 2025 · 4 comments · May be fixed by #12611
Assignees
Labels
🐞 bug Something isn't working

Comments

@Moiyu1998
Copy link

Moiyu1998 commented Jan 8, 2025

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.15.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

The chunk size of the knowledge base is limited to 4000. I changed the environment variable to 8191, but it didn't take effect; the limit is still 4000. In version 0.14.1, modifying this variable used to work.The size limit for both parent and child chunks is also 4000.
image
image
image

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

Copy link

dosubot bot commented Jan 8, 2025

I found a similar unsolved discussion regarding an issue with modifying the maximum chunk length through environment variables. You can view it here [1].

In your case, it seems that the environment variable change to increase the chunk size limit from 4000 to 8191 is not taking effect in version 0.15.0, whereas it worked in version 0.14.2. This might indicate a change in how environment variables are handled between these versions. Unfortunately, I couldn't find specific documentation or code comments explaining the process of updating configuration settings through environment variables in the current version.

You might want to check if there are any recent changes in the codebase related to the chunk size configuration or how environment variables are applied. If the issue persists, consider reviewing the code to ensure that the environment variable is correctly set and applied in the configuration.

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other

@dosubot dosubot bot added the 🐞 bug Something isn't working label Jan 8, 2025
@Moiyu1998
Copy link
Author

I found a similar unsolved discussion regarding an issue with modifying the maximum chunk length through environment variables. You can view it here [1].

In your case, it seems that the environment variable change to increase the chunk size limit from 4000 to 8191 is not taking effect in version 0.15.0, whereas it worked in version 0.14.2. This might indicate a change in how environment variables are handled between these versions. Unfortunately, I couldn't find specific documentation or code comments explaining the process of updating configuration settings through environment variables in the current version.

You might want to check if there are any recent changes in the codebase related to the chunk size configuration or how environment variables are applied. If the issue persists, consider reviewing the code to ensure that the environment variable is correctly set and applied in the configuration.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other

Sorry, I made a mistake. It is version 0.14.1.

@crazywoola
Copy link
Member

crazywoola commented Jan 8, 2025

export const MaxLengthInput: FC<InputNumberProps> = (props) => {
const { t } = useTranslation()
return <FormField label={<div className='system-sm-semibold mb-1'>
{t('datasetCreation.stepTwo.maxLength')}
</div>}>
<InputNumber
type="number"
className='h-9'
placeholder={'≤ 4000'}
max={4000}
min={1}
{...props}
/>
</FormField>
}

@crazywoola crazywoola assigned WTW0313 and unassigned JohnJyong Jan 8, 2025
@WTW0313 WTW0313 assigned iamjoel and unassigned WTW0313 Jan 9, 2025
@Moiyu1998
Copy link
Author

Moiyu1998 commented Jan 9, 2025

export const MaxLengthInput: FC<InputNumberProps> = (props) => {
const { t } = useTranslation()
return <FormField label={<div className='system-sm-semibold mb-1'>
{t('datasetCreation.stepTwo.maxLength')}
</div>}>
<InputNumber
type="number"
className='h-9'
placeholder={'≤ 4000'}
max={4000}
min={1}
{...props}
/>
</FormField>
}

Sorry, modifying this file does not take effect. Due to network restrictions, I am unable to rebuild the image. Is there any other way to resolve this issue? Alternatively, could this issue be fixed as soon as possible?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐞 bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants