Writer api

[sandorkertesz]

This PR

Adds a new writer API:

• Target
• Encoder
• to_target()
• create_target()
• create_encoder()

https://earthkit-data--596.org.readthedocs.build/en/596/guide/targets.html
https://earthkit-data--596.org.readthedocs.build/en/596/guide/encoders.html

Deprecates:

• write() and save() method
• new_grib_output(), new_grib_coder()
• readers.grib.output.GribOutput, readers.grib.output.GribCoder

Removes:

• writers.GribWriter

Some counter-intuitive features

When we call to_target() it does the following:

• creates a target
• writes the data into the target
• calls close() on the target

This has implications in the "file" target. Currently, when we call FileTarget.close() it behaves differently whether the target was created with a file path or a file pointer:

• if the FileTarget was created with a path close() closes the associated file pointer
• if the FileTarget was created with a file pointer close() does not close the associated file pointer

Examples (assume ds, ds1 and ds2 are fieldlists)

# this opens the target file and closes it 
ds.to_target("file", "my.grib") 

fp = open("my_file", "wb")
ds1.to_target("file", fp)  # does not close fp
ds2.to_target("file", fp) # does not close fp
# fp is still open at this point, we need to close it explicitly
close(fp)

fp = open("my_file", "wb")
with create_target("file", fp) as t:
    for d in ds:
        t.write(data=d)

   # this will not close fp
   t.close()

# when t goes out of scope the context manager calls 
# close() which does not close fp
# fp is still open at this point, we need to close it explicitly
close(fp)

If we closed the user provided file pointer when close() is called this code would not work:

fp = open("my_file", "wb")
ds1.to_target("file", fp)  # close fp
ds2.to_target("file", fp) # ERROR: fp closed, cannot write to it

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 87.93970% with 72 lines in your changes missing coverage. Please review.

Project coverage is 90.02%. Comparing base (a8f4437) to head (633b88d).

Files with missing lines	Patch %	Lines
tests/targets/test_target_fdb.py	40.00%	30 Missing ⚠️
tests/plugins/test_encoders_plugin.py	44.82%	15 Missing and 1 partial ⚠️
tests/targets/test_target_file.py	91.44%	13 Missing ⚠️
tests/grib/test_grib_output.py	92.51%	2 Missing and 9 partials ⚠️
tests/sources/test_cds.py	80.00%	1 Missing ⚠️
tests/sources/test_mars.py	80.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #596      +/-   ##
===========================================
- Coverage    90.28%   90.02%   -0.27%     
===========================================
  Files          153      156       +3     
  Lines        11565    11957     +392     
  Branches       546      583      +37     
===========================================
+ Hits         10442    10764     +322     
- Misses         953     1013      +60     
- Partials       170      180      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[chpolste]

Calling to_netcdf without a path restricts the data format to NetCDF3, btw:

If no path is provided, this function returns the resulting netCDF file as bytes; in this case, we need to use scipy, which does not support netCDF version 4 (the default format becomes NETCDF3_64BIT).

https://docs.xarray.dev/en/stable/generated/xarray.Dataset.to_netcdf.html

[sandorkertesz]

@chpolste, thanks for pointing this out. Do you know if there is a way to generate netCDF version 4 as bytes from xarray?