Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random-access for variable-based encoding (i.e. use SetStepSelection for ADIOS2 steps) #1706

Open
wants to merge 10 commits into
base: dev
Choose a base branch
from

Conversation

franzpoeschel
Copy link
Contributor

@franzpoeschel franzpoeschel commented Dec 16, 2024

TODO:

  • MPI-aware preparsing
  • Probably for a later PR: Preparsing also in READ_LINEAR (i.e. eliminate duplicate Iterations better than now)
  • Edge case: What if a particle species completely vanishes after some iteration, making it appear as an empty, i.e. constant component -> need to distinguish this better in the frontend
  • Better testing
  • Documentation, examples
  • Replace group-based encoding with variable-based encoding more thoroughly for ADIOS2 (maybe follow-up)
  • Can we make variable-based encoding the default output for ADIOS2? (maybe follow-up for easier reversibility)
  • Check if the engine supports this mode
  • Check if the datasets have the same number of steps as there are snapshots
  • hehe maybe pre-parse the entire thing in linear encoding and only then switch to random-access?

@franzpoeschel franzpoeschel added backend: ADIOS2 api: new additions to the API labels Dec 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: new additions to the API backend: ADIOS2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant