Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Idefics_FT.ipynb: Size of the dataset #13

Open
joris-sense opened this issue Sep 10, 2024 · 0 comments
Open

Idefics_FT.ipynb: Size of the dataset #13

joris-sense opened this issue Sep 10, 2024 · 0 comments

Comments

@joris-sense
Copy link

joris-sense commented Sep 10, 2024 •
edited
Loading

Hi, I uploaded a slightly modified dataset for my own experiment here and its size is only about 1 GB, not 3.5 GB (as your version here), even though it seems to have the same number of entries and the images should have the same size. It seems to me that 1 GB is more realistic because there are 20k entries and a typical image size from those I sampled is about 50 KB. Is it possible that your dataset contains some inaccessible rows?
@joris-sense joris-sense changed the title Idefics_FT.ipynb: Size of your dataset Idefics_FT.ipynb: Size of the dataset Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@joris-sense