Added functionality to test multiple versions of python using tox #994

Scott-Simmons · 2022-12-24T06:31:43Z

What functionality changed?

This PR adds functionality to be able to locally test multiple versions of python using tox.

This PR will run the test suite for python versions 3.7, 3.8, 3.9, 3.10, and 3.11 for whichever platform the user runs the tests on. The specific microversion of python depends on what is installed on the user's machine.

tox will skip over running the test suite for a given version if it cannot find a particular python environment.

How to run?

To test on multiple python versions, edit envlist in setup.cfg to include the python versions you want to test on, and then run:

tox -r

in the top-level directory for tsfresh. Currently the versions being tested span between python 3.7-3.11

Why?

This PR will make it much easier to (locally) test tsfresh against multiple versions of python, for a given platform

What code changed?

setup.cfg was changed to include some extra configuration information.

tox was added to test-requirements.txt

Tips for reviewing/running

By default, tox will look in binary directories for any relevant python interpreters. For example, if

envlist = (py37, py38)

exists in setup.cfg, then tox will look inside binary directories for python executables named similar to python3.7.X and python 3.8.X. If it cannot find any executables for a given python version, then it will skip over testing that version.

Note that in this PR:

envlist = py37, py38, py39, py310, py311

But it should also be noted that while included in the envlist, py311 is currently not compatible with tsfresh due to a numba dependency conflict.

Using `pyenv` in tandem with `tox`

A recommended way to handle multiple versions of python is with pyenv. pyenv will allow you to install multiple versions of python in a well-organised fashion.

If you choose to use pyenv to manage the different python versions installed on your machine, then the executables of each python version will be in ~/.pyenv/shims/ which will not be found by tox by default. A recommended solution to this is to make a .python-version file, with the versions of python that you want tox to look for.

For example, if the output of running

pyenv versions

shows that you have installed python 3.7.16 and python 3.8.16, then you should put 3.7.16 and 3.8.16 as separate lines in the .python-version file. Running tox -r will then run the tests for python3.7.16 and python3.8.16, and tox will know where to find the relevant python interpreters.

`pyenv` important note

The python version that the tox command is initially invoked from matters!

Running pyenv which python will show the version of python from which the tox command will be invoked from. If tox is initially invoked from a version of python that is not supported by the package (i.e. package is invoked from python 3.6 and python_requires is >=3.7), then tox will fail for all environments, including python versions that would otherwise work if tox was originally invoked from a version of python supported with the package.

Note that we can still test tsfresh on unsupported versions of python (such as 3.6), provided that tox is initially invoked from a version of python that is in tsfresh python_requires (such as 3.8).

Log files

Log files are stored in the .tox directory which is created once tox is run.

codecov-commenter · 2022-12-24T06:34:00Z

Codecov Report

Base: 93.42% // Head: 93.42% // No change to project coverage 👍

Coverage data is based on head (42b5dd3) compared to base (6714034).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #994   +/-   ##
=======================================
  Coverage   93.42%   93.42%           
=======================================
  Files          20       20           
  Lines        1886     1886           
  Branches      371      371           
=======================================
  Hits         1762     1762           
  Misses         85       85           
  Partials       39       39

Impacted Files	Coverage Δ
tsfresh/feature_extraction/feature_calculators.py	`92.46% <100.00%> (ø)`
tsfresh/transformers/feature_selector.py	`98.14% <100.00%> (ø)`

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Scott-Simmons · 2022-12-26T01:38:19Z

The main changes are in setup.cfg and .gitignore. The changes to the other files are style changes in order to pass the pre-commit hook.

setup.cfg

Scott-Simmons · 2024-11-04T11:12:48Z

@nils-braun is there still a need for this PR or should it be closed? Happy to pick it up again and get it over the line (if needed)

Scott-Simmons · 2024-11-04T11:13:32Z

If so, will clean up the commits too...

nils-braun

🙀 I am so sorry @Scott-Simmons - your PR just slipped through and I never ever had a look into it. Really sorry, this is not a good experience for you. Thanks for taking the time. Please do ping me earlier next time :)

I like the changes. If you update to the newest version of main, the pre-commit changes should also not be need anymore. After that, I think the changes are already ready to be merged. Do you think it makes sense to also add parts of your PR description into the how_to_contribute.rst document?

setup.cfg

nils-braun · 2024-11-05T20:41:59Z

setup.cfg

+    python -m pip install -r requirements.txt
+    python -m pip install -r test-requirements.txt
+    python -m pip install -r optional-requirements.txt
+    python -m pip install -r rdocs-requirements.txt


Why do we need to install the docs requirements for the tests?

Addressed f848737

Adds tox support for testing against multiple versions of python

Scott-Simmons · 2024-11-10T11:34:05Z

Really sorry, this is not a good experience for you

No apologies needed, all good with me. Only recently remembered this.

Do you think it makes sense to also add parts of your PR description into the how_to_contribute.rst document?

If I understand you correctly, I've added something to that effect here

Error: This request has been automatically failed because it uses a deprecated version of actions/upload-artifact: v1. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/

Looks like the builds are failing for unrelated reason w.r.t this PR. Can look at updating the actions file in a new PR.

Add example of how to raise a PR.

Scott-Simmons · 2024-11-10T11:44:55Z

Looks like the builds are failing for unrelated reason w.r.t this PR. Can look at updating the actions file in a new PR.

@nils-braun marking as ready for review despite the above

Scott-Simmons

Unsure this PR is needed anymore since we have https://github.com/blue-yonder/tsfresh/blob/1297c8ca5bd6f8f23b4de50e3f052fb4ec1307f8/.github/workflows/test.yml#

nils-braun · 2024-11-11T19:43:16Z

Unsure this PR is needed anymore since we have https://github.com/blue-yonder/tsfresh/blob/1297c8ca5bd6f8f23b4de50e3f052fb4ec1307f8/.github/workflows/test.yml#

Yes, definitely your PR ist still needed. The file you point to is for CI/CD, but it does not help the average developer. We have this matrix since the start of the library, but we never had it usable for developers. So your change is still needed!

nils-braun · 2024-11-11T19:51:36Z

If I understand you correctly, I've added something to that effect here

Ah sorry, what you did is also great, but I actually meant adding the description on how to run the tox tests into the document (basically the tox -r). But the description on how to do PRs is also good (and your is a good example!)

…-versions

nils-braun

Looks good now @Scott-Simmons!
You can or can not add the description on the tox -r command - I leave it up to you. I will merge the PR tomorrow :)
Thanks again for the contribution!

Adds a comment about how to run tests locally on multiple versions of python. Ensures it is clear that it only works if particular version of python is installed on the users machine. Recommends `pytest` as a possible way to manage multiple local versions of python. Note the hyperlink out to `setup.cfg` is pinned to commit sha 1297c8c instead of pinning to main branch.

Scott-Simmons · 2024-11-12T08:17:13Z

You can or can not add the description on the tox -r command

Added in 1e9c3bc

but we never had it usable for developers

Was thinking about this, maybe we could have a docker container for local testing as well? To get around the developer needing to have python versions 3.7, ...3.12 on their development machine.

Something like:

makefile

TEST_DOCKERFILE := Dockerfile-test
TEST_CONTAINER := test-tsfresh
test-all:
    docker build -f $(TEST_DOCKERFILE) -t $(TEST_CONTAINER) .
    docker run --rm $(TEST_CONTAINER)

Dockerfile-test

FROM python:3.10-slim # could be something else too...

# Haven't tested this probably more dependencies required...
RUN apt-get update && \
    apt-get install -y python3.7 python3.8 python3.9 python3.10 python3.11 python3.12

RUN pip install tox

# Just an idea may not need to copy whole thing in....
WORKDIR /app
COPY . .

CMD ["tox -r"]

@nils-braun if this is something that looks useful I can take this on - I don't think it will be too much work but could be useful for developer experience?

Scott-Simmons marked this pull request as draft December 26, 2022 01:45

Scott-Simmons marked this pull request as ready for review December 26, 2022 04:41

Scott-Simmons commented Dec 26, 2022

View reviewed changes

setup.cfg Outdated Show resolved Hide resolved

nils-braun force-pushed the feature/software-testing-for-multiple-python-versions branch from e381ced to 42b5dd3 Compare February 21, 2023 21:59

nils-braun reviewed Nov 5, 2024

View reviewed changes

Scott-Simmons marked this pull request as draft November 10, 2024 09:37

Scott-Simmons force-pushed the feature/software-testing-for-multiple-python-versions branch from eee6081 to 1e9f9d1 Compare November 10, 2024 09:40

Adding tox support

f848737

Adds tox support for testing against multiple versions of python

Scott-Simmons force-pushed the feature/software-testing-for-multiple-python-versions branch from 1e9f9d1 to f848737 Compare November 10, 2024 11:11

PR etiquette comment

b7de7d0

Add example of how to raise a PR.

Scott-Simmons force-pushed the feature/software-testing-for-multiple-python-versions branch from a590777 to b7de7d0 Compare November 10, 2024 11:42

Scott-Simmons marked this pull request as ready for review November 10, 2024 11:44

Scott-Simmons commented Nov 10, 2024

View reviewed changes

Merge branch 'main' into feature/software-testing-for-multiple-python…

f01c2fc

…-versions

nils-braun approved these changes Nov 11, 2024

View reviewed changes

Scott-Simmons mentioned this pull request Nov 14, 2024

Dockerise testing for dev #1093

Merged

nils-braun merged commit ece0714 into blue-yonder:main Nov 14, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added functionality to test multiple versions of python using tox #994

Added functionality to test multiple versions of python using tox #994

Scott-Simmons commented Dec 24, 2022 •

edited

Loading

codecov-commenter commented Dec 24, 2022 •

edited

Loading

Scott-Simmons commented Dec 26, 2022

Scott-Simmons commented Nov 4, 2024

Scott-Simmons commented Nov 4, 2024

nils-braun left a comment

nils-braun Nov 5, 2024

Scott-Simmons Nov 10, 2024 •

edited

Loading

Scott-Simmons commented Nov 10, 2024 •

edited

Loading

Scott-Simmons commented Nov 10, 2024

Scott-Simmons left a comment

nils-braun commented Nov 11, 2024

nils-braun commented Nov 11, 2024

nils-braun left a comment

Scott-Simmons commented Nov 12, 2024

Added functionality to test multiple versions of python using tox #994

Added functionality to test multiple versions of python using tox #994

Conversation

Scott-Simmons commented Dec 24, 2022 • edited Loading

What functionality changed?

How to run?

Why?

What code changed?

Tips for reviewing/running

Using pyenv in tandem with tox

pyenv important note

Log files

codecov-commenter commented Dec 24, 2022 • edited Loading

Codecov Report

Scott-Simmons commented Dec 26, 2022

Scott-Simmons commented Nov 4, 2024

Scott-Simmons commented Nov 4, 2024

nils-braun left a comment

Choose a reason for hiding this comment

nils-braun Nov 5, 2024

Choose a reason for hiding this comment

Scott-Simmons Nov 10, 2024 • edited Loading

Choose a reason for hiding this comment

Scott-Simmons commented Nov 10, 2024 • edited Loading

Scott-Simmons commented Nov 10, 2024

Scott-Simmons left a comment

Choose a reason for hiding this comment

nils-braun commented Nov 11, 2024

nils-braun commented Nov 11, 2024

nils-braun left a comment

Choose a reason for hiding this comment

Scott-Simmons commented Nov 12, 2024

Scott-Simmons commented Dec 24, 2022 •

edited

Loading

Using `pyenv` in tandem with `tox`

`pyenv` important note

codecov-commenter commented Dec 24, 2022 •

edited

Loading

Scott-Simmons Nov 10, 2024 •

edited

Loading

Scott-Simmons commented Nov 10, 2024 •

edited

Loading