AttributeError: 'list' object has no attribute 'items' in _pytest.runner.SetupState.setup

[pawamoy]

Upon running my test suite, I sometimes get these errors:

pytest -c config/pytest.ini -n auto  tests

============================= test session starts ==============================
platform linux -- Python 3.11.5, pytest-7.4.2, pluggy-1.3.0
Using --randomly-seed=196610002
rootdir: /media/data/dev/aria2p/config
configfile: pytest.ini
plugins: cov-4.1.0, xdist-3.3.1, anyio-3.7.1, rerunfailures-9.1.1, randomly-3.15.0
created: 6/6 workers
6 workers [480 items]

......................................................................... [ 15%]
........................................................................ [ 30%]
...........................................................x............ [ 45%]
........................................................................ [ 60%]
........................................................................ [ 75%]
........................................................................ [ 90%]
..............................................R                          [100%]R [100%]R [100%]R [100%]R [100%]E [100%]Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>

==================================== ERRORS ====================================
___________________ ERROR at setup of test_pause_subcommand ____________________
[gw2] linux -- Python 3.11.5 /usr/bin/python3.11

cls = <class '_pytest.runner.CallInfo'>
func = <function call_runtest_hook.<locals>.<lambda> at 0x7f8f124a5e40>
when = 'setup'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: "Callable[[], TResult]",
        when: "Literal['collect', 'setup', 'call', 'teardown']",
        reraise: Optional[
            Union[Type[BaseException], Tuple[Type[BaseException], ...]]
        ] = None,
    ) -> "CallInfo[TResult]":
        """Call func, wrapping the result in a CallInfo.
    
        :param func:
            The function to call. Called without arguments.
        :param when:
            The phase in which the function is called.
        :param reraise:
            Exception or exceptions that shall propagate if raised by the
            function, instead of being wrapped in the CallInfo.
        """
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: Optional[TResult] = func()

__pypackages__/3.11/lib/_pytest/runner.py:341: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
__pypackages__/3.11/lib/_pytest/runner.py:262: in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
__pypackages__/3.11/lib/pluggy/_hooks.py:493: in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
__pypackages__/3.11/lib/pluggy/_manager.py:115: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
__pypackages__/3.11/lib/_pytest/runner.py:157: in pytest_runtest_setup
    item.session._setupstate.setup(item)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_pytest.runner.SetupState object at 0x7f8f19b3a6d0>
item = <Function test_pause_subcommand>

    def setup(self, item: Item) -> None:
        """Setup objects along the collector chain to the item."""
        needed_collectors = item.listchain()
    
        # If a collector fails its setup, fail its entire subtree of items.
        # The setup is not retried for each item - the same exception is used.
>       for col, (finalizers, exc) in self.stack.items():
E       AttributeError: 'list' object has no attribute 'items'

__pypackages__/3.11/lib/_pytest/runner.py:484: AttributeError
----------------------------- Captured stderr call -----------------------------
GID#0000000000000001 cannot be paused now
=========================== short test summary info ============================
ERROR config/test_cli.py::test_pause_subcommand - AttributeError: 'list' object has no attribute 'items'
============== 478 passed, 1 xfailed, 1 error, 5 rerun in 20.20s ===============

The error happens on random tests (not always the same ones), sometimes on a single test, sometimes on many more.

My repo: https://github.com/pawamoy/aria2p.
My config:

[pytest]
norecursedirs =
  .git
  .tox
  .env
  dist
  build
python_files =
  test_*.py
  *_test.py
  tests.py
addopts =
  --cov
  --cov-append
  --cov-config config/coverage.ini
  --randomly-dont-reset-seed
  --reruns 5
  --reruns-delay 0.1
testpaths =
  tests

# action:message_regex:warning_class:module_regex:line
filterwarnings =
  error
  # TODO: remove once pytest-xdist 4 is released
  ignore:.*rsyncdir:DeprecationWarning:xdist
  ignore:.*the imp module:DeprecationWarning:future

Operating system: Linux.
Pytest and plugins:

pytest-cov==4.1.0
pytest-randomly==3.15.0
pytest-rerunfailures==9.1.1
pytest-xdist==3.3.1
pytest==7.4.2

All packages

-e file:///media/data/dev/aria2p#egg=aria2p
annotated-types==0.6.0
ansimarkup==1.5.0
anyio==3.7.1
appdirs==1.4.4
asciimatics==1.14.0
Babel==2.13.0
beautifulsoup4==4.12.2
black==23.9.1
blacken-docs==1.16.0
certifi==2023.7.22
charset-normalizer==3.3.0
click==8.1.7
colorama==0.4.6
coverage==7.3.2
csscompressor==0.9.5
dparse==0.6.3
duty==1.0.0
editables==0.5
execnet==2.0.2
failprint==1.0.2
fastapi==0.103.2
future==0.18.3
ghp-import==2.1.0
git-changelog==2.3.1
gitdb==4.0.10
GitPython==3.1.37
griffe==0.36.5
h11==0.14.0
htmlmin2==0.1.13
idna==3.4
iniconfig==2.0.0
Jinja2==3.1.2
jsmin==3.0.1
loguru==0.7.2
lxml==4.9.3
markdown-callouts==0.3.0
markdown-exec==1.6.0.1.0.1
Markdown==3.5
MarkupSafe==2.1.3
mergedeep==1.3.4
mkdocs-autorefs==0.5.0
mkdocs-coverage==1.0.0
mkdocs-gen-files==0.5.0
mkdocs-git-committers-plugin-2==1.2.0
mkdocs-literate-nav==0.6.1
mkdocs-material-extensions==1.2
mkdocs-material==9.4.5
mkdocs-minify-plugin==0.7.1
mkdocs==1.5.3
mkdocstrings-python==1.7.3.1.5.1
mkdocstrings==0.23.0
mypy-extensions==1.0.0
mypy==1.5.1
packaging==23.2
paginate==0.5.6
pathspec==0.11.2
Pillow==10.0.1
platformdirs==3.11.0
pluggy==1.3.0
psutil==5.9.5
ptyprocess==0.7.0
pydantic-core==2.10.1
pydantic==2.4.2
pyfiglet==0.8.post1
Pygments==2.16.1
pymdown-extensions==10.3
pyperclip==1.8.2
pytest-cov==4.1.0
pytest-randomly==3.15.0
pytest-rerunfailures==9.1.1
pytest-xdist==3.3.1
pytest==7.4.2
python-dateutil==2.8.2
pyyaml-env-tag==0.1
PyYAML==6.0.1
regex==2023.10.3
requests==2.31.0
responses==0.23.3
ruamel-yaml-clib==0.2.8
ruamel-yaml==0.17.35
ruff==0.0.292
safety==2.3.4
semver==3.0.2
setuptools==68.2.2
six==1.16.0
smmap==5.0.1
sniffio==1.3.0
soupsieve==2.5
starlette==0.27.0
toml==0.10.2
types-Markdown==3.5.0.0
types-PyYAML==6.0.12.12
types-requests==2.31.0.8
types-setuptools==68.2.0.0
types-toml==0.10.8.7
typing-extensions==4.8.0
urllib3==2.0.6
uvicorn==0.23.2
watchdog==3.0.0
wcwidth==0.2.8
websocket-client==1.6.4

---

I tried disabling/enabling plugins to see from which it could come, and it seems to only happen when rerunfailures is enabled. Besides, I see code in it that could explain the error:

def _remove_failed_setup_state_from_session(item):
    """
    Note: remove all _prepare_exc attribute from every col in stack of
          _setupstate and cleaning the stack itself
    """
    prepare_exc = "_prepare_exc"
    setup_state = getattr(item.session, "_setupstate")
    for col in setup_state.stack:
        if hasattr(col, prepare_exc):
            delattr(col, prepare_exc)
    setup_state.stack = list()         # <------ here

The error seems to happen when a test fails and therefore the rerun machinery kicks in.

[ShuGuangTR]

Hi @pawamoy , I had the same problem, but I solved it.
I rolled back pytest 7.4.2---->5.2.1 , and then, is worked!
If you need reruns, pls try.

[icemac]

Maybe there way a change in pytest, which we do not trigger in our own tests.
I'd be happy to see a PR to fix this issue. I am not sure whether it could be as easy as replacing the list with a dict.

[ShuGuangTR]

Maybe there way a change in pytest, which we do not trigger in our own tests. I'd be happy to see a PR to fix this issue. I am not sure whether it could be as easy as replacing the list with a dict.

Thanks for your answer, that's right, I saw it's fixed on 3acc7b3
Upgrading the pytest-rerunfailures to last version can solve this issue

=========== in pytest-rerunfailures #245 commit =======================================

    --setup_state.stack = []
    ++setup_state.stack = {}

[pawamoy]

Great, closing then! Thanks everyone!