Add pytest fixture to clean up the IO loop

[0x26res]

I recently came across this error in my unit tests:

RuntimeError: This event loop is already running

The error doesn't come from streamz, it's from jupyter_core, in this recent change.

But it is triggered because my streamz tests don't clean up properly after them selves.
They leave the global async event loop, asyncio.get_event_loop(), in a running state.

To fix it I had to add this fixture to my unit tests:

@pytest.fixture(autouse=True)
def loop_cleaner():
    yield None
    close_io_loop()


def close_io_loop():

    IOLoop().current().stop()
    IOLoop().current().close()
    assert not asyncio.get_event_loop().is_running()
    asyncio.get_event_loop().close()
    assert asyncio.get_event_loop().is_closed()
    asyncio.set_event_loop(None)

It's a bit similar to the existing pristine_loop, but doesn't exactly do the same thing (afaict).

I hope this can help any one having similar issues.

[martindurant]

If it were a scope="session" fixture, I see no reason it shouldn't be added to conftest. Some of these issues may still be coming from the mixture of tornado and asyncio styles.

[0x26res]

@martindurant sorry I wasn't clear. I'm not suggesting we add this fixture to conftest.

I'm suggesting we put it in unit_test.py the same way there is the clean fixture. So users of streamz can use that new fixture in their unit tests. I for example use the clean fixture in some of my tests already.

[amotl]

Hi there,

we are not 100% sure, but we think we have been tripped by the same issue when conceiving LorryStream.

We have been able to work around it by leveraging the pytest-asyncio-cooperative package. A corresponding example test case is demonstrated at ¹.

With kind regards,
Andreas.

Footnotes

1. https://github.com/daq-tools/lorrystream/blob/b0412d7/tests/test_engine.py#L40-L58 ↩

[martindurant]

I'm suggesting we put it in unit_test.py

Certainly that would be fine too. Also, a little documentation around this, as linked by @amotl would make sense, for anyone else that might be doing this kind of thing.