Improve coverage integration

Changes.md

@@ -2,6 +2,14 @@
 
 Note: development is tracked on the [`develop` branch](https://github.com/chmp/ipytest/tree/develop).
 
+## `0.14.2`
+
+- Support collecting branch coverage in notebooks (e.g., via `--cov--branch`)
+- Add `ipytest.autoconfig(coverage=True)` to simplify using `pytest-cov` inside
+  notebooks
+- Add experimental `ipytest.cov.translate_cell_filenames()` to simplify
+  interpretation of collected coverage information
+
 ## `0.14.1`
 
 - Add a [Coverage.py](https://coverage.readthedocs.io/en/latest/index.html)

Readme.md

@@ -99,22 +99,23 @@ this case, using [`ipytest.run()`][ipytest.run] and
 | [`ipytest.cov`](#ipytestcov)
 
 <!-- minidoc "function": "ipytest.autoconfig", "header_depth": 3 -->
-### `ipytest.autoconfig(rewrite_asserts=<default>, magics=<default>, clean=<default>, addopts=<default>, run_in_thread=<default>, defopts=<default>, display_columns=<default>, raise_on_error=<default>)`
+### `ipytest.autoconfig(rewrite_asserts=<default>, magics=<default>, clean=<default>, addopts=<default>, run_in_thread=<default>, defopts=<default>, display_columns=<default>, raise_on_error=<default>, coverage=<default>)`
 
-[ipytest.autoconfig]: #ipytestautoconfigrewrite_assertsdefault-magicsdefault-cleandefault-addoptsdefault-run_in_threaddefault-defoptsdefault-display_columnsdefault-raise_on_errordefault
+[ipytest.autoconfig]: #ipytestautoconfigrewrite_assertsdefault-magicsdefault-cleandefault-addoptsdefault-run_in_threaddefault-defoptsdefault-display_columnsdefault-raise_on_errordefault-coveragedefault
 
 Configure `ipytest` with reasonable defaults.
 
 Specifically, it sets:
 
-* `rewrite_asserts`: `True`
-* `magics`: `True`
-* `clean`: `'[Tt]est*'`
 * `addopts`: `('-q', '--color=yes')`
-* `run_in_thread`: `False`
+* `clean`: `'[Tt]est*'`
+* `coverage`: `False`
 * `defopts`: `'auto'`
 * `display_columns`: `100`
+* `magics`: `True`
 * `raise_on_error`: `False`
+* `rewrite_asserts`: `True`
+* `run_in_thread`: `False`
 
 See [`ipytest.config`][ipytest.config] for details.
 
@@ -169,9 +170,9 @@ inside a CI/CD context, use `ipytest.autoconfig(raise_on_error=True)`.
 <!-- minidoc -->
 
 <!-- minidoc "function": "ipytest.config", "header_depth": 3 -->
-### `ipytest.config(rewrite_asserts=<keep>, magics=<keep>, clean=<keep>, addopts=<keep>, run_in_thread=<keep>, defopts=<keep>, display_columns=<keep>, raise_on_error=<keep>)`
+### `ipytest.config(rewrite_asserts=<keep>, magics=<keep>, clean=<keep>, addopts=<keep>, run_in_thread=<keep>, defopts=<keep>, display_columns=<keep>, raise_on_error=<keep>, coverage=<default>)`
 
-[ipytest.config]: #ipytestconfigrewrite_assertskeep-magicskeep-cleankeep-addoptskeep-run_in_threadkeep-defoptskeep-display_columnskeep-raise_on_errorkeep
+[ipytest.config]: #ipytestconfigrewrite_assertskeep-magicskeep-cleankeep-addoptskeep-run_in_threadkeep-defoptskeep-display_columnskeep-raise_on_errorkeep-coveragedefault
 
 Configure `ipytest`
 
@@ -186,6 +187,12 @@ The following settings are supported:
 * `rewrite_asserts` (default: `False`): enable ipython AST transforms
   globally to rewrite asserts
 * `magics` (default: `False`): if set to `True` register the ipytest magics
+* `coverage` (default: `False`): if `True` configure `pytest` to collect
+  coverage information. This functionality requires the `pytest-cov` package
+  to be installed. It adds `--cov --cov-config={GENERATED_CONFIG}` to the
+  arguments when invoking `pytest`. **WARNING**: this option will hide
+  existing coverage configuration files. See [`ipytest.cov`](#ipytestcov)
+  for details
 * `clean` (default: `[Tt]est*`): the pattern used to clean variables
 * `addopts` (default: `()`): pytest command line arguments to prepend to
   every pytest invocation. For example setting
@@ -217,9 +224,9 @@ The following settings are supported:
 The return code of the last pytest invocation.
 
 <!-- minidoc "function": "ipytest.run", "header_depth": 3 -->
-### `ipytest.run(*args, module=None, plugins=(), run_in_thread=<default>, raise_on_error=<default>, addopts=<default>, defopts=<default>, display_columns=<default>)`
+### `ipytest.run(*args, module=None, plugins=(), run_in_thread=<default>, raise_on_error=<default>, addopts=<default>, defopts=<default>, display_columns=<default>, coverage=<default>)`
 
-[ipytest.run]: #ipytestrunargs-modulenone-plugins-run_in_threaddefault-raise_on_errordefault-addoptsdefault-defoptsdefault-display_columnsdefault
+[ipytest.run]: #ipytestrunargs-modulenone-plugins-run_in_threaddefault-raise_on_errordefault-addoptsdefault-defoptsdefault-display_columnsdefault-coveragedefault
 
 Execute all tests in the passed module (defaults to `__main__`) with pytest.
 
@@ -320,7 +327,7 @@ plugins =
     ipytest.cov
 ```
 
-With this config file, the coverage can be collected using
+With this config file, coverage information can be collected using
 [pytest-cov][ipytest-cov-pytest-cov] with
 
 ```python
@@ -330,9 +337,35 @@ def test():
     ...
 ```
 
+`ipytest.autoconfig(coverage=True)` automatically adds the `--cov` flag and the
+path of a generated config file to the Pytest invocation. In this case no
+further configuration is required.
+
+There are some known issues of `ipytest.cov`
+
+- Each notebook cell is reported as an individual file
+- Lines that are executed at import time may not be encountered in tracing and
+  may be reported as not-covered (One example is the line of a function
+  definition)
+- Marking code to be excluded in branch coverage is currently not supported
+  (incl. coveragepy pragmas)
+
 [coverage-py-config-docs]: https://coverage.readthedocs.io/en/latest/config.html
 [ipytest-cov-pytest-cov]: https://pytest-cov.readthedocs.io/en/latest/config.html
 
+#### `ipytest.cov.translate_cell_filenames(enabled=True)`
+
+[ipytest.cov.translate_cell_filenames]: #ipytestcovtranslate_cell_filenamesenabledtrue
+
+Translate the filenames of notebook cells in coverage information.
+
+If enabled, `ipytest.cov` will translate the temporary file names generated
+by ipykernel (e.g, `ipykernel_24768/3920661193.py`) to their cell names
+(e.g., `In[6]`).
+
+**Warning**: this is an experimental feature and not subject to any
+stability guarantees.
+
 <!-- minidoc -->
 
 ## Development

ipytest/_config.py

@@ -5,25 +5,27 @@
 default_clean = "[Tt]est*"
 
 defaults = {
-    "rewrite_asserts": True,
-    "magics": True,
-    "clean": default_clean,
     "addopts": ("-q", "--color=yes"),
-    "run_in_thread": False,
+    "clean": default_clean,
+    "coverage": False,
     "defopts": "auto",
     "display_columns": 100,
+    "magics": True,
     "raise_on_error": False,
+    "rewrite_asserts": True,
+    "run_in_thread": False,
 }
 
 current_config = {
-    "rewrite_asserts": False,
-    "magics": False,
-    "clean": default_clean,
     "addopts": (),
-    "run_in_thread": False,
+    "clean": default_clean,
+    "coverage": False,
     "defopts": "auto",
     "display_columns": 100,
+    "magics": False,
     "raise_on_error": False,
+    "rewrite_asserts": False,
+    "run_in_thread": False,
 }
 
 _rewrite_transformer = None
@@ -67,6 +69,7 @@ def autoconfig(
     defopts=default,
     display_columns=default,
     raise_on_error=default,
+    coverage=default,
 ):
     """Configure `ipytest` with reasonable defaults.
 
@@ -91,6 +94,7 @@ def config(
     defopts=keep,
     display_columns=keep,
     raise_on_error=keep,
+    coverage=default,
 ):
     """Configure `ipytest`
 
@@ -105,6 +109,12 @@ def config(
     * `rewrite_asserts` (default: `False`): enable ipython AST transforms
       globally to rewrite asserts
     * `magics` (default: `False`): if set to `True` register the ipytest magics
+    * `coverage` (default: `False`): if `True` configure `pytest` to collect
+      coverage information. This functionality requires the `pytest-cov` package
+      to be installed. It adds `--cov --cov-config={GENERATED_CONFIG}` to the
+      arguments when invoking `pytest`. **WARNING**: this option will hide
+      existing coverage configuration files. See [`ipytest.cov`](#ipytestcov)
+      for details
     * `clean` (default: `[Tt]est*`): the pattern used to clean variables
     * `addopts` (default: `()`): pytest command line arguments to prepend to
       every pytest invocation. For example setting

ipytest/_impl.py

@@ -4,6 +4,7 @@
 import importlib
 import os
 import pathlib
+import re
 import shlex
 import sys
 import threading
@@ -28,6 +29,7 @@ def run(
     addopts=default,
     defopts=default,
     display_columns=default,
+    coverage=default,
 ):
     """Execute all tests in the passed module (defaults to `__main__`) with pytest.
 
@@ -64,6 +66,7 @@ def run(
     addopts = default.unwrap(addopts, current_config["addopts"])
     defopts = default.unwrap(defopts, current_config["defopts"])
     display_columns = default.unwrap(display_columns, current_config["display_columns"])
+    coverage = default.unwrap(coverage, current_config["coverage"])
 
     if module is None:
         import __main__ as module
@@ -77,6 +80,7 @@ def run(
         addopts=addopts,
         defopts=defopts,
         display_columns=display_columns,
+        coverage=coverage,
     )
 
     ipytest.exit_code = exit_code
@@ -250,13 +254,18 @@ def force_reload(*include: str, modules: Optional[Dict[str, ModuleType]] = None)
         modules.pop(name, None)
 
 
-def _run_impl(*args, module, plugins, addopts, defopts, display_columns):
+def _run_impl(*args, module, plugins, addopts, defopts, display_columns, coverage):
     with _prepared_env(module, display_columns=display_columns) as filename:
-        full_args = _build_full_args(args, filename, addopts=addopts, defopts=defopts)
+        full_args = _build_full_args(
+            args, filename, addopts=addopts, defopts=defopts, coverage=coverage
+        )
+        if coverage:
+            warn_for_existing_coverage_configs()
+
         return pytest.main(full_args, plugins=[*plugins, FixProgramNamePlugin()])
 
 
-def _build_full_args(args, filename, *, addopts, defopts):
+def _build_full_args(args, filename, *, addopts, defopts, coverage):
     arg_mapping = ArgMapping(
         # use basename to ensure --deselect works
         # (see also: https://github.com/pytest-dev/pytest/issues/6751)
@@ -266,7 +275,16 @@ def _build_full_args(args, filename, *, addopts, defopts):
     def _fmt(arg):
         return arg.format_map(arg_mapping)
 
+    if coverage:
+        import ipytest.cov
+
+        coverage_args = ("--cov", f"--cov-config={ipytest.cov.config_path}")
+
+    else:
+        coverage_args = ()
+
     all_args = [
+        *coverage_args,
         *(_fmt(arg) for arg in addopts),
         *(_fmt(arg) for arg in args),
     ]
@@ -521,3 +539,44 @@ def is_notebook_node_id(prev: Optional[str], arg: str) -> bool:
     return all(
         not is_notebook_node_id(prev, arg) for prev, arg in zip([None, *args], args)
     )
+
+
+def warn_for_existing_coverage_configs():
+    if configs := find_coverage_configs("."):
+        print(
+            "Warning: found existing coverage.py configuration in "
+            f"{[p.name for p in configs]}. "
+            "These config files are ignored when using "
+            "`ipytest.autoconfig(coverage=True)`."
+            "Consider adding the `ipytest.cov` plugin directly to the config "
+            "files and adding  `--cov` to the `%%ipytest` invocation.",
+            file=sys.stderr,
+        )
+
+
+def find_coverage_configs(root):
+    root = pathlib.Path(root)
+
+    result = []
+    if (p := root.joinpath(".coveragerc")).exists():
+        result.append(p)
+
+    result += _find_files_with_lines(root, ["setup.cfg", "tox.ini"], r"^\[coverage:.*$")
+    result += _find_files_with_lines(root, ["pyproject.toml"], r"^\[tool\.coverage.*$")
+
+    return result
+
+
+def _find_files_with_lines(root, paths, pat):
+    for path in paths:
+        path = root.joinpath(path)
+        if path.exists():
+            try:
+                with open(path, "rt") as fobj:
+                    for line in fobj:
+                        if re.match(pat, line) is not None:
+                            yield path
+                            break
+
+            except Exception:
+                pass