Fix: CodeCov's omitted files show up in report

[dominiquesydow]

Description

CodeCov's omitted files partially show (e.g. files in kissim/tests and kissim._version.py). Find out why and exclude them.

Todos

• Test [no success]: Explicitly omit file that shows up in CodeCov but shouldn't (use as example kissim/tests/conftest.py). > no success
• Test [no success]: Add --cov-config=setup.cfg to pytest command in .github/workflows/CI.yaml > no success; leave it in anyways
• Test [no success]: Pin pytest to version 5.* > no success
• Test [success]: Use */_version.py (asterisk!) instead of kissim/_version.py in .github/workflows/CI.yaml > successful, no clue why this was an issue here but not in other repos. No clue.

Questions

Todos are in this case my questions :)

Status

• Ready to go

[dominiquesydow]

Hopefully fixes #43

[codecov-io]

Codecov Report

Merging #43 (91923ac) into master (cf404b6) will increase coverage by 12.41%.
The diff coverage is n/a.

[dominiquesydow]

The following files should be omitted as defined in setup.cfg:

kissim/setup.cfg

Lines 5 to 9 in cf404b6

	omit =
	# Omit the tests
	kissim/tests/*
	# Omit generated versioneer
	kissim/_version.py

But the following files show up in the CodeCov report:

https://codecov.io/gh/volkamerlab/kissim/tree/7bf58aafc71aedc733310a1e00a1cc9e2a53bdd9/kissim

Observations:

• In kissim/tests interestingly only the files are included (not the subfolders).
• Also, interestingly, with this commit all lines in kissim._version.py are included, before at least some lines were excluded. Such a behaviour has happened before, see here: Add tests for CLI #42 (comment)

[dominiquesydow]

Try this suggestion by @j-wags (add --cov-config=setup.cfg to pytest command in .github/workflows/CI.yaml): MolSSI/cookiecutter-cms#91

[dominiquesydow]

Lastest commit not analyzed by CodeCov

[dominiquesydow]

Latest CodeCov report: https://codecov.io/gh/volkamerlab/kissim/tree/10024c2bb65a90e1a9d09d6c6f81a4d0822aacac/kissim

Problem remains:

[dominiquesydow]

Dropping this here - CI uses

pytest                    6.2.2            py37hf985489_0    conda-forge
pytest-cov                2.11.1             pyh44b312d_0    conda-forge

[dominiquesydow]

Compared this repo to another repo where the same setup.cfg and CI.yaml (w.r.t. to the pytest commands) files are used:
https://github.com/dominiquesydow/dynophores

There we use:

pytest                    5.4.3            py37hc8dfbb8_0    conda-forge
pytest-cov                2.11.1             pyh44b312d_0    conda-forge

pytest is pinned to version 5.*
https://github.com/dominiquesydow/dynophores/blob/07e8747394010556c076f42eb68e65e729d31cf8/devtools/conda-envs/test_env.yaml#L21

Try that out.

[dominiquesydow]

The kissim/tests folder is omitted:
https://codecov.io/gh/volkamerlab/kissim/tree/e6ee7bbfa9bacffdadd16b49e7ff57530a2cffaf/kissim

kissim/_version.py is still there.

I am confused.

[dominiquesydow]

@jaimergp could you please take a look at this PR - do you have any more pointers for me?

[jaimergp]

Can you try using your codecov.yml file instead? https://docs.codecov.io/docs/ignoring-paths

[dominiquesydow]

A bit embarressed to admit but I only just now understood that I am dealing with a truly two-step procedure here:

1. Generate coverage.xml file with coverage.py
   

   kissim/.github/workflows/CI.yaml

   Lines 54 to 58 in 91923ac

   	- name: Run tests
   	# conda setup requires this special shell
   	shell: bash -l {0}
   	run: |
   	pytest -v --cov=kissim --cov-report=xml --cov-config=setup.cfg --color=yes kissim/tests/

2. Upload the coverage.xml file to CodeCov
   

   kissim/.github/workflows/CI.yaml

   Lines 59 to 64 in 91923ac

   	- name: CodeCov
   	uses: codecov/codecov-action@v1
   	with:
   	file: ./coverage.xml
   	flags: unittests
   	name: codecov-${{ matrix.os }}-py${{ matrix.python-version }}

   • https://github.com/codecov/codecov-action

   • When using pytest, use https://pypi.org/project/pytest-cov/ with the XML report. This report output is detected and uploaded by the Codecov upload script. https://docs.codecov.io/docs/python

---

That tells me finding the problem should not start with step 2 (CodeCov) but step 1.

So I ran locally
pytest -v --cov=kissim --cov-config=setup.cfg kissim/tests/

and figured out that I have to adjust kissim/_version.py to */_version.py

kissim/.github/workflows/CI.yaml

Lines 54 to 58 in 91923ac

	- name: Run tests
	# conda setup requires this special shell
	shell: bash -l {0}
	run: |
	pytest -v --cov=kissim --cov-report=xml --cov-config=setup.cfg --color=yes kissim/tests/

so that the coverage looks correct locally (no kissim/_version.py, nothing from kissim/tests/):

----------- coverage: platform linux, python 3.8.6-final-0 -----------
Name                                                  Stmts   Miss  Cover
-------------------------------------------------------------------------
kissim/__init__.py                                        5      0   100%
kissim/api/__init__.py                                    2      0   100%
kissim/api/compare.py                                    27      0   100%
kissim/api/encode.py                                     16      0   100%
kissim/cli/__init__.py                                    2      0   100%
kissim/cli/compare.py                                     8      0   100%
kissim/cli/encode.py                                     15      0   100%
kissim/cli/main.py                                       19      0   100%
kissim/cli/utils.py                                      17      0   100%
kissim/comparison/__init__.py                             4      0   100%
kissim/comparison/feature_distances.py                   76      0   100%
kissim/comparison/feature_distances_generator.py         94      0   100%
kissim/comparison/fingerprint_distance.py                39      1    97%
kissim/comparison/fingerprint_distance_generator.py     139      7    95%
kissim/comparison/utils.py                               26      0   100%
kissim/definitions.py                                    10      0   100%
kissim/encoding/__init__.py                               4      0   100%
kissim/encoding/base.py                                  73      0   100%
kissim/encoding/features/__init__.py                      5      0   100%
kissim/encoding/features/base.py                         17      0   100%
kissim/encoding/features/exposure.py                     70      1    99%
kissim/encoding/features/sco.py                          54      0   100%
kissim/encoding/features/sitealign.py                    54      0   100%
kissim/encoding/features/subpockets.py                   92      0   100%
kissim/encoding/fingerprint.py                           50      0   100%
kissim/encoding/fingerprint_generator.py                113      1    99%
kissim/encoding/fingerprint_normalized.py                57      0   100%
kissim/io/__init__.py                                     3      0   100%
kissim/io/biopython.py                                  179      8    96%
kissim/io/data.py                                        74      1    99%
kissim/io/dataframe.py                                   10      0   100%
kissim/schema.py                                          7      0   100%
kissim/utils.py                                          27      0   100%
-------------------------------------------------------------------------
TOTAL                                                  1388     19    99%

Next is step 2 - does CodeCov show the same? Waiting for the CI.

[dominiquesydow]

CodeCov for last commit:
https://codecov.io/gh/volkamerlab/kissim/tree/91923ac5f9db1b133ff5a918860c559965c5aed4/kissim

kissim/_version.py and kissim/tests/* are gone! :)

Differences from local coverage:

• cli module: 100% coverage locally, with GHA 19 missing lines - full main.py, means that subprocesses do not work?
• utils module: 100% coverage locally, with GHA 2 lines missing - no sure why yet, see utils.py report

[dominiquesydow]

Merging this PR, since problem resolved. Attend to coverage improvements elsewhere.