Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General tidy up of data writing #36

Merged
merged 24 commits into from
Mar 8, 2022
Merged

General tidy up of data writing #36

merged 24 commits into from
Mar 8, 2022

Conversation

noemifrisina
Copy link
Collaborator

@noemifrisina noemifrisina commented Feb 14, 2022

TODO list to start from:

  • Clean up CLI
  • Add Eiger/Tristan-like masks to add gaps to blank data
  • Switch to direct_chunk_write to speed up
  • Fix fake events writer
  • Add pseudo events generator
  • Separated vds writer for images
  • Probably needs vds also for events (Am I really sure I want to do this?)

This was linked to issues Feb 14, 2022
@noemifrisina
Copy link
Collaborator Author

Log of event generation for one chunk of events for a Tristan2M detector:

2022-02-25 15:06:57,656 NeXusGenerator.writer.data INFO Start generating one chunk of pseudo events for (1, 2) modules of Tristan 2M
2022-02-25 15:08:14,632 NeXusGenerator.writer.data INFO Time taken to generate pseudo-event list: 76.91 s.

@noemifrisina
Copy link
Collaborator Author

VDS could still do with some improvement (especially after unexpected error on I24).

@noemifrisina noemifrisina merged commit 5240b9c into main Mar 8, 2022
@noemifrisina noemifrisina deleted the speed-up branch March 8, 2022 09:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Generate pseudo events Speed up data writing
1 participant