Skip to content

Commit

Permalink
DOC: update developer guide on Handling Dependencies
Browse files Browse the repository at this point in the history
  • Loading branch information
neutrinoceros committed Sep 11, 2024
1 parent 5ad93c4 commit 342340c
Showing 1 changed file with 24 additions and 31 deletions.
55 changes: 24 additions & 31 deletions doc/source/developing/testing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -441,54 +441,47 @@ Handling yt Dependencies
------------------------

Our dependencies are specified in ``pyproject.toml``. Hard dependencies are found in
``options.install_requires``, while optional dependencies are specified in
``options.extras_require``. The ``full`` target contains the specs to run our
``project.dependencies``, while optional dependencies are specified in
``project.optional-dependencies``. The ``full`` target contains the specs to run our
test suite, which are intended to be as modern as possible (we don't set upper
limits to versions unless we need to). The ``minimal`` target is used to check
that we don't break backward compatibility with old versions of upstream
projects by accident. It is intended to pin strictly our minimal supported
versions. The ``test`` target specifies the tools needed to run the tests, but
limits to versions unless we need to).

The ``test`` target specifies the tools needed to run the tests, but
not needed by yt itself.

Documentation and typechecking requirements are found in ``requirements/``,
and used in ``tests/ci_install.sh``.

**Python version support.**
When a new Python version is released, it takes about
a month or two for yt to support it, since we're dependent on bigger projects
like numpy and matplotlib. We vow to follow numpy's deprecation plan regarding
our supported versions for Python and numpy, defined formally in `NEP 29
<https://numpy.org/neps/nep-0029-deprecation_policy.html>`_. However, we try to
avoid bumping our minimal requirements shortly before a yt release.
We vow to follow numpy's deprecation plan regarding our supported versions for Python
and numpy, defined formally in
`NEP 29 <https://numpy.org/neps/nep-0029-deprecation_policy.html>`_, but generally
support larger version intervals than recommended in this document.

**Third party dependencies.**
We attempt to make yt compatible with a wide variety of upstream software
versions.
However, sometimes a specific version of a project that yt depends on
causes some breakage and must be blacklisted in the tests or a more
experimental project that yt depends on optionally might change sufficiently
that the yt community decides not to support an old version of that project.

**Note.**
Some of our optional dependencies are not trivial to install and their support
may vary across platforms. To manage such issue, we currently use requirement
files in additions to ``pyproject.toml``. They are found in
``tests/*requirements.txt`` and used in ``tests/ci_install.sh``.

We attempt to make yt compatible with a wide variety of upstream software
versions. However, sometimes a specific version of a project that yt depends on
causes some breakage and must be blacklisted in the tests or a more
experimental project that yt depends on optionally might change sufficiently
that the yt community decides not to support an old version of that project.

To handle cases like this, the versions of upstream software projects installed
on the machines running the yt test suite are pinned to specific version
numbers that must be updated manually. This prevents breaking the yt tests when
a new version of an upstream dependency is released and allows us to manage
updates in upstream projects at our pace.
may vary across platforms.

If you would like to add a new dependency for yt (even an optional dependency)
or would like to update a version of a yt dependency, you must edit the
``tests/test_requirements.txt`` file, this path is relative to the root of the
repository. This file contains an enumerated list of direct dependencies and
pinned version numbers. For new dependencies, simply append the name of the new
``pyproject.toml`` file. For new dependencies, simply append the name of the new
dependency to the end of the file, along with a pin to the latest version
number of the package. To update a package's version, simply update the version
number in the entry for that package.

Finally, we also run a set of tests with "minimal" dependencies installed. When adding tests that depend on an optional dependency, you can wrap the test with the ``yt.testing.requires_module decorator`` to ensure it does not run during the minimal dependency tests (see yt/frontends/amrvac/tests/test_read_amrvac_namelist.py for a good example). If for some reason you need to update the listing of packages that are installed for the "minimal" dependency tests, you will need to edit ``tests/test_minimal_requirements.txt``.
Finally, we also run a set of tests with "minimal" dependencies installed.
When adding tests that depend on an optional dependency, you can wrap the test
with the ``yt.testing.requires_module decorator`` to ensure it does not run
during the minimal dependency tests (see
``yt/frontends/amrvac/tests/test_read_amrvac_namelist.py`` for a good example).
If for some reason you need to update the listing of packages that are installed
for the "minimal" dependency tests, you will need to update
``requirements/minimal_env.txt``.

0 comments on commit 342340c

Please sign in to comment.