import ValidationError from asdf, drop jsonschema as a dependency

[braingram]

asdf has recently run into compatibility issues with jsonschema 4.18+. The newer versions of jsonschema dropped support for a feature required in asdf which left us with little choice but to vendorize jsonschema. This presented an opportunity to clean up some leaky bits of the asdf API including the issue that jsonschema.ValidationError was transparently passed to user code and libraries (like dkist) had to import ValidationError from jsonschema.

asdf 2.15 introduced ValidationError as part of the public api available at asdf.exceptions.ValidationError. This PR changes the import ValidationError from asdf.exceptions (instead of jsonschema).

asdf 2.15.1 includes the vendorized jsonschema, keeps jsonschema as a dependency and attempts to use the exceptions from the installed jsonschema (to not break some downstream libraries that use asdf). We are planning to drop jsonschema as a dependency and stop using it's exceptions in asdf 3.0 (or possibly in 2.15.2 if further changes to jsonschema make it's exceptions incompatible).

Please see the What's New page in the asdf 2.15.1 docs for more information and let me know if you have any questions, comments or concerns.

[codecov]

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (9696c7c) 98.28% compared to head (22171f9) 98.28%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #295   +/-   ##
=======================================
  Coverage   98.28%   98.28%           
=======================================
  Files          40       40           
  Lines        2451     2452    +1     
=======================================
+ Hits         2409     2410    +1     
  Misses         42       42           

Files	Coverage Δ
dkist/dataset/loader.py	100.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Cadair]

Thanks a lot for the PR. I think this is my main issue with this, I am not sure it's really practical for us to pull in the very latest version of asdf.

Does asdf throw a version warning if you load a file with new asdf or just third party plugins?

[braingram]

Thanks for taking and look and for the response.

I tested reading and writing asdf files across versions (2.9.2 and 2.15.0). In both cases the file was produced with:

t = astropy.time.Time(1.0, format='mjd')
af = asdf.AsdfFile({'t': t})
af.write_to('time.asdf')

For 2.9.2 I installed asdf-astropy==0.1.1 (to match the minimum pin in this package). For 2.15.0 I installed asdf-astropy==0.4.0.

Opening the file written with asdf 2.9.2 in asdf 2.15.0 produced no warnings as all the extensions used to produce the file were installed and were at least the same (or in this case newer) versions.

Opening the file written with asdf 2.15.0 in asdf 2.9.2 produced two warnings, one for the asdf-astropy extension and one for the builtin extension that handles the asdf core objects:

/Users/bgraham/.pyenv/versions/3.10.6/envs/dkist_asdf_downstream/lib/python3.10/site-packages/asdf/asdf.py:348: AsdfWarning: File 'file:///Users/bgraham/Desktop/time.asdf' was created with extension class 'asdf.extension.BuiltinExtension' (from package asdf==2.15.0), but older package (asdf==2.9.2) is installed.
  warnings.warn(msg, AsdfWarning)
/Users/bgraham/.pyenv/versions/3.10.6/envs/dkist_asdf_downstream/lib/python3.10/site-packages/asdf/asdf.py:348: AsdfWarning: File 'file:///Users/bgraham/Desktop/time.asdf' was created with extension URI 'asdf://asdf-format.org/core/extensions/core-1.5.0' (from package asdf-astropy==0.4.0), but older package (asdf-astropy==0.1.1) is installed.
  warnings.warn(msg, AsdfWarning)

I'm trying to think of a way to not pin asdf to at least 2.15.0. I can think of three options:

1. Drop jsonschema as a dependency and import ValidationError from the top level asdf. It appears that this was available at least as far back as 2.9. I should note that having ValidationError exposed at the top level asdf will likely be deprecated in asdf 3.x.x and removed in 4.0.0. It's probably ambitious to say 4.0.0 will come out in the next 6 months (and as we don't yet have a concrete roadmap for the included changes it will likely take longer) but the deprecation of asdf.ValidationError in favor of asdf.exceptions.ValidationError will come much sooner and likely soon after the 3.0.0 release (which I'm hoping will happen in the next few months at the latest). This deprecation warning can always be ignored so this does seem like an ok option for keeping the lower pin at 2.9.2 and dropping jsonschema.

2. Keep jsonschema as a dependency and change the ValidationError imports to first try from asdf.exceptions and then fall back to jsonschema.

3. Drop jsonschema and change the imports to first try from asdf.exceptions and then from asdf. I kind of like this option the best but I'm happy to make whatever changes you'd like.

I will update this PR for option 3 above but please let me know if you'd rather go with one of the other options or if you have other plans for keeping the pin. Thanks!