You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
xarray.Dataset.to_netcdf has an option for encoding, allowing data to be written as int16, saving 75% of disk space compared to direct storage of double and 50% compared to float. Compression can additionally be applied for further savings.
The encoding requires an offset and scale factor to be chosen for each variable, so that the values can be stored as
The (potential) loss of precision is usually no problem for atmospheric variables (most reanalysis data is delivered like this anyway), as long as the scale factor and offset are appropriately chosen. This is where some specific knowledge about sensible ranges for each field stored is required.
Users could always choose their own, but we might want to provide a default set of encodings for the output variables of the QGField properties. They need to be general enough to match any season, hemisphere, region, etc. but specific enough to retain precision. I suggest we work out a set of values in this issue and ship these conveniently with the package eventually.
xarray.Dataset.to_netcdf has an option for encoding, allowing data to be written as int16, saving 75% of disk space compared to direct storage of double and 50% compared to float. Compression can additionally be applied for further savings.
The encoding requires an offset and scale factor to be chosen for each variable, so that the values can be stored as
The (potential) loss of precision is usually no problem for atmospheric variables (most reanalysis data is delivered like this anyway), as long as the scale factor and offset are appropriately chosen. This is where some specific knowledge about sensible ranges for each field stored is required.
Users could always choose their own, but we might want to provide a default set of encodings for the output variables of the QGField properties. They need to be general enough to match any season, hemisphere, region, etc. but specific enough to retain precision. I suggest we work out a set of values in this issue and ship these conveniently with the package eventually.
See also:
The text was updated successfully, but these errors were encountered: