-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Performance painfully slow exporting large number of inspection-items #209
Comments
Hi @jashwood, I'd like to understand more about your setup. What kind of machine/platform are you running on? |
We are running on Windows Server 2019, 20 core IaaS VM in SQL Azure, with 140GB of ram and a 10Gbps internet connection, and very high performance disks |
We have the same issue (with 535k inspection items), on both linux x64 and darwin ARM. Query time gets linearly slower and slower for each batch that is written out so there is some inefficiency with the SQLite queries. e.g. on a high performance Mac Studio, export takes 30 minutes. Our deployment environment allocates significantly lower IOPS and takes over 5 hours. Example output from a dev machine:
|
@OscarBarrett thanks for raising this! If you have any bandwidth, we'd love to see contributions to addressing this. |
Hey there,
We have over 100,000 inspection_item records.. exporting these to csv takes over an hour, sometimes more.
Can the performance be improved somehow?
The text was updated successfully, but these errors were encountered: