More difficulties in 0.23 with async fixtures

[bmerry]

This might have the same underlying cause as #705 / #706, but I didn't see the specific error message I'm running into so I thought I should open a separate bug.

To reproduce:

conftest.py (copied verbatim from https://pytest-asyncio.readthedocs.io/en/latest/how-to-guides/run_session_tests_in_same_loop.html):

import pytest

from pytest_asyncio import is_async_test


def pytest_collection_modifyitems(items):
    pytest_asyncio_tests = (item for item in items if is_async_test(item))
    session_scope_marker = pytest.mark.asyncio(scope="session")
    for async_test in pytest_asyncio_tests:
        async_test.add_marker(session_scope_marker)

foo/__init__.py: empty

foo/test_foo.py:

import pytest_asyncio


@pytest_asyncio.fixture(scope="package")
async def bar() -> int:
    return 3

async def test_stuff(bar: int) -> None:
    pass

Gives this error:

==================================== ERRORS ====================================
_________________________ ERROR at setup of test_stuff _________________________
file /home/bmerry/work/sdp/bugs/pytest-asyncio-session/foo/test_foo.py, line 8
  async def test_stuff(bar: int) -> None:
      pass
file /home/bmerry/work/sdp/bugs/pytest-asyncio-session/foo/test_foo.py, line 4
  @pytest_asyncio.fixture(scope="package")
  async def bar() -> int:
      return 3
E       fixture 'foo/__init__.py::<event_loop>' not found
>       available fixtures: __pytest_repeat_step_number, _session_event_loop, anyio_backend, anyio_backend_name, anyio_backend_options, bar, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, check, class_mocker, cov, doctest_namespace, event_loop, event_loop_policy, foo/test_foo.py::<event_loop>, mocker, module_mocker, monkeypatch, no_cover, package_mocker, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, session_mocker, testrun_uid, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, unused_tcp_port, unused_tcp_port_factory, unused_udp_port, unused_udp_port_factory, worker_id
>       use 'pytest --fixtures [testpath]' for help on them.

Python 3.10.12, pytest-asyncio 0.23.2.

[bmerry]

I forgot to include the pytest.ini:

[pytest]
asyncio_mode = auto

[seifertm]

Thanks for the reproducer. Good call for opening a separate issue!

[nzeid]

Let me know if this isn't the best place to post this, but I have a gut feeling it's very closely related to the sloppiness of fixture scopes and event loop initialization.

I was busting my head all day trying to refactor my code to get rid of the def event_loop override per version 0.23. I could not for the life of me get function-scoped asyncio fixtures to work on session-scoped asyncio test functions with simple markers/decorators. I would almost always end up with the dreaded MultipleEventLoopsRequestedError. I can understand why multiple event loops are prohibited, but I can't understand why fixtures renewing at different cadences condemns them to live in a separate event loop. This is absolutely a must-have.

For anyone who happens upon this, the workaround for now is as follows:

# This allows all fixtures to access the session-scoped event loop:
@pytest_asyncio.fixture(scope="session")
async def global_event_loop():
    return asyncio.get_running_loop()


# The function of the async fixture you want to create, but without the marker:
async def clear_db():
    async with Transaction() as tx:
        await tx.execute(sql.text("DELETE FROM some_table"))


# Use this non-async fixture to mimic the behavior of the async fixture you need.
# Run the async code using global_event_loop.
@pytest.fixture(autouse=True, scope="function")
def clear_db_wrapper(global_event_loop):
    yield
    global_event_loop.run_until_complete(clear_db())

(Edit: Fixed and clarified code.)

[seifertm]

@nzeid I'm sorry that you had to go through so much trouble. Pytest-asyncio v0.23 supports running tests in different event loops. There's a loop for each level of the test suite (session, package, module, class, function). However, pytest-asyncio wrongly assumes that the scope of an async fixture is the same as the scope of the event loop in which the fixture runs. This essentially makes it impossible to have a session-scoped fixture that runs in the same loop as a function-scoped test. See #706.

[av223119]

I'm experiencing the similar issue, which I believe is another manifestation of #706.

Namely, the error message fixture 'test_something.py::<event_loop>' not found which, unexpectedly, is generated when tests from test_something_else.py being run. The only thing these two modules have in common is that they use one shared module-scoped fixture from conftest.py. It appears that pytest-asyncio tried to use the event loop from the first module while running the second one. It looks awfully similar to #829 which, in turn, I believe is just another manifestation of #706