Enabling compression for the ADIOS2 backend #1556
-
I want to use openPMD with ADIOS2 as back-end, and I want to enable ADIOS2 compression via the blosc library. How do I do this? When I create an ADIOS2 file directly the I would write approximately this: const adios2::Operator compressor =
adios.DefineOperator("Blosc2Compressor", adios2::ops::LosslessBlosc);
// Use a high compression rate and a bitshuffle filter
const adios2::Params compressor_options{
{adios2::ops::blosc::key::clevel, adios2::ops::blosc::key::clevel_9},
{adios2::ops::blosc::key::doshuffle,
adios2::ops::blosc::key::doshuffle_bitshuffle},
...
adios2::Variable<CCTK_REAL> var =
io.DefineVariable<CCTK_REAL>(varname, {}, {},
{1, 1, 1});
var.AddOperation(compresser, compresser_options); |
Beta Was this translation helpful? Give feedback.
Answered by
franzpoeschel
Nov 15, 2023
Replies: 1 comment
-
Hey Erik, Check also the documentation. |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
eschnett
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hey Erik,
these parameters are passed via JSON/TOML, see e.g. here for zlib or here for Blosc.
You can specify the default compression by passing the parameter to the
Series
constructor or a dataset-specific compression, by setting this JSON value in theDataset
class (as passed toresetDataset()
.Check also the documentation.