create_blocking returns a function that blocks when called from a coroutine function

[spencerwilson]

Hello 👋

Setup

Adding some assert statements to the code in the 0.6.6. README:

import asyncio

from synchronicity import Synchronizer

synchronizer = Synchronizer()

@synchronizer.create_blocking
async def f(x):
    await asyncio.sleep(1.0)
    return x**2


# Running f in a synchronous context blocks until the result is available
assert type(f(42)) == int


async def g():
    # Running f in an asynchronous context works the normal way
    ret = f(42)
    assert type(await ret) == int

asyncio.run(g())

Expected behavior

• No assertion errors.
• f returns an int when called from a sync context
• f returns an Awaitable[int] when called from a coroutine function

Actual behavior

Running this on synchronicity 0.6.6 in Python 3.11 one observes the following:

Traceback (most recent call last):
  File "/test.py", line 23, in <module>
    asyncio.run(g())
  File "/opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/test.py", line 20, in g
    assert type(await ret) == int
                ^^^^^^^^^
TypeError: object int can't be used in 'await' expression

More info

It seems that when calling a wrapped function from inside a coroutine function (in the example case: calling f from g), the wrapped function does not return a coroutine as suggested; instead the wrapped function blocks and returns the non-coroutine value. AFAICT it does this because it takes a code path that leads here:

synchronicity/synchronicity/synchronizer.py

Line 307 in daa99f3

value = fut.result()

The prose in the README also suggests that when calling a wrapped function from a coroutine function, the value returned should be an Awaitable:

When you call anything, it will detect if you're running in a synchronous or asynchronous context, and behave correspondingly.

In the synchronous case, it will simply block until the result is available (note that you can make it return a future as well, see later)
In the asynchronous case, it works just like the usual business of calling asynchronous code

It's curious because the README also documents how one can pass _future=True to a wrapped function to coax it into returning an awaitable. It's a bit confusing that this is opt in after having just read both the sample code (which throws the above TypeError) and the words "In the asynchronous case, it works just like the usual business of calling asynchronous code".

[spencerwilson]

Another gotcha: When using _future=True the object returned is a concurrent.futures.Future, which is not an awaitable. This type is distinct from asyncio.Future, which is awaitable.

The former can in most cases be adapted into the latter using asyncio.wrap_future. The complete working code looks like:

import asyncio

from synchronicity import Synchronizer

synchronizer = Synchronizer()

@synchronizer.create_blocking
async def f(x):
    await asyncio.sleep(1.0)
    return x**2


# Running f in a synchronous context blocks until the result is available
assert type(f(42)) == int


async def g():
    # Running f in an asynchronous context works the normal way
    ret = asyncio.wrap_future(f(42, _future=True))
    assert type(await ret) == int

asyncio.run(g())

[freider]

Hi! Sorry for the very late reply to this.
The documentation has been quite outdated and I just pushed a new PR that puts the README more in line with how the current version of synchronicity works.

It no longer tries to auto-infer sync vs async return values based on if it runs within an event loop or not, and instead leaves it to the user to explicitly call either wrapped_function(...) (synchronous) or wrapped_function.aio(...) (async) for the two use cases. This allows for much better static typing support etc.

As for the _future=True return values, the idea is that it should mostly (if at all) be used from non-async contexts where you don't have other primitives that allow same-thread concurrency. If you are running within an async context you could probably just as well use the async interface (.aio) directly instead and rely on asyncio for gathering multiple tasks etc.

Hope this helps, if you're still a user of the library :)
I'll make sure to be more responsive to reported issues here in the future