Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maximum number of products parsed per file? #74

Open
minouye opened this issue Feb 1, 2019 · 1 comment
Open

Maximum number of products parsed per file? #74

minouye opened this issue Feb 1, 2019 · 1 comment

Comments

@minouye
Copy link

minouye commented Feb 1, 2019

Is there a maximum number of products that can be parsed in a single file? I'm running into a limit at around 30,000 listings (for a single file with ~1M products).

I've tried parsing separate sections of the file to figure out if there is some kind of formatting that's causing artificial breakpoints. But I've been unable to find any irregularities in the data.

require 'im_onix'

feed = ONIX::ONIXMessage.new
feed.parse("feed.xml")
puts feed.products.count
@julbouln
Copy link
Member

julbouln commented Feb 3, 2020

Since im_onix use a DOM parser, everything is loaded in memory, so the limitation will come from your machine's memory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants