Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement]: When manually setting table.event-time-field to a field, it should widen its data file mertic length #1604

Open
3 tasks done
shendanfengg opened this issue Jun 25, 2023 · 2 comments

Comments

@shendanfengg
Copy link
Contributor

Search before asking

  • I have searched in the issues and found no similar issues.

What would you like to be improved?

The current default length of the iceberg table for metadata metric is 'write.metadata.metrics.default' = 'truncate(16)'. meaning that data larger than 16 will have a loss of precision.
When we set table.event-time-field we often choose a time field in the format yyyy-MM-dd HH:mm:ss, obviously this string is longer than 16, then there will be a loss of precision when parsing the upperbounds and lowerbounds of the datafile.
From the user's point of view, most of the cases will not be able to change 'write.metadata.metrics.default' to a larger value or to 'full', so it is necessary to help the user to set this configuration.

How should we improve?

AMS should set global default configuration such as 'write.metadata.metrics.default' to 'truncate(32)' when creating a table, or just for the fields set in the table.event-time-field configuration mertric changes

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

Copy link

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

@github-actions github-actions bot added the stale label Aug 20, 2024
@zhoujinsong zhoujinsong added good first issue Good for newcomers not-stale and removed stale labels Aug 20, 2024
@MarigWeizhi
Copy link
Contributor

If there is no assignee to this issue, can I try it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants