Skip to content

Commit

Permalink
deprecate --trio200-blocking-calls, replacing it with --async200-bloc…
Browse files Browse the repository at this point in the history
…king-calls (#251)

* deprecate --trio200-blocking-calls, replacing it with --async200-blocking-calls

* Various small improvements to docs/usage.rst
  • Loading branch information
jakkdl authored May 22, 2024
1 parent c656315 commit 9653fde
Show file tree
Hide file tree
Showing 9 changed files with 173 additions and 47 deletions.
5 changes: 5 additions & 0 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ Changelog

*[CalVer, YY.month.patch](https://calver.org/)*

24.5.3
======
- Rename config option ``trio200-blocking-calls`` to :ref:`async200-blocking-calls`.
- ``trio200-blocking-calls`` is now deprecated.

24.5.2
======
- ASYNC101 now also warns on anyio & asyncio taskgroups.
Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ to likely performance issues, to minor points of idiom that might signal
a misunderstanding.


The plugin may well be too noisy or pedantic depending on your requirements or opinions, in which case you should consider :ref:`--disable` for those rules.
The plugin may well be too noisy or pedantic depending on your requirements or opinions, in which case you should consider :ref:`disable` for those rules.
Pairs well with flake8-bugbear.


Expand Down
22 changes: 22 additions & 0 deletions docs/rules.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,13 @@ General rules
- **ASYNC110**: ``while <condition>: await [trio/anyio].sleep()`` should be replaced by a ``[trio/anyio].Event``.
- **ASYNC111**: Variable, from context manager opened inside nursery, passed to ``start[_soon]`` might be invalidly accessed while in use, due to context manager closing before the nursery. This is usually a bug, and nurseries should generally be the inner-most context manager.
- **ASYNC112**: Nursery body with only a call to ``nursery.start[_soon]`` and not passing itself as a parameter can be replaced with a regular function call.

.. _async113:

- **ASYNC113**: Using :meth:`trio.Nursery.start_soon` in ``__aenter__`` doesn't wait for the task to begin. Consider replacing with ``nursery.start``.

.. _async114:

- **ASYNC114**: Startable function (i.e. has a ``task_status`` keyword parameter) not in ``--startable-in-context-manager`` parameter list, please add it so ASYNC113 can catch errors when using it.
- **ASYNC115**: Replace ``[trio/anyio].sleep(0)`` with the more suggestive ``[trio/anyio].lowlevel.checkpoint()``.
- **ASYNC116**: ``[trio/anyio].sleep()`` with >24 hour interval should usually be ``[trio/anyio].sleep_forever()``.
Expand All @@ -33,6 +39,7 @@ Blocking sync calls in async functions

Note: 22X, 23X and 24X has not had asyncio-specific suggestions written.

.. _async200:

- **ASYNC200**: User-configured error for blocking sync calls in async functions. Does nothing by default, see :ref:`async200-blocking-calls` for how to configure it.
- **ASYNC210**: Sync HTTP call in async function, use ``httpx.AsyncClient``. This and the other ASYNC21x checks look for usage of ``urllib3`` and ``httpx.Client``, and recommend using ``httpx.AsyncClient`` as that's the largest http client supporting anyio/trio.
Expand All @@ -55,11 +62,26 @@ Optional rules disabled by default
.. _async900:

- **ASYNC900**: Async generator without ``@asynccontextmanager`` not allowed. You might want to enable this on a codebase since async generators are inherently unsafe and cleanup logic might not be performed. See https://github.com/python-trio/flake8-async/issues/211 and https://discuss.python.org/t/using-exceptiongroup-at-anthropic-experience-report/20888/6 for discussion.

.. _async910:

- **ASYNC910**: Exit or ``return`` from async function with no guaranteed checkpoint or exception since function definition. You might want to enable this on a codebase to make it easier to reason about checkpoints, and make the logic of ASYNC911 correct.

.. _async911:

- **ASYNC911**: Exit, ``yield`` or ``return`` from async iterable with no guaranteed checkpoint since possible function entry (yield or function definition)
Checkpoints are ``await``, ``async for``, and ``async with`` (on one of enter/exit).
- **ASYNC912**: A timeout/cancelscope has checkpoints, but they're not guaranteed to run. Similar to ASYNC100, but it does not warn on trivial cases where there is no checkpoint at all. It instead shares logic with ASYNC910 and ASYNC911 for parsing conditionals and branches.

.. _autofix-support:

Autofix support
===============
The following rules support :ref:`autofixing <autofix>`.
- ASYNC100
- ASYNC910
- ASYNC911

Removed rules
================

Expand Down
121 changes: 91 additions & 30 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@
Installation
************

.. code-block:: console
Install from `PyPI <https://pypi.org/project/flake8-async>`__

.. code-block:: sh
pip install flake8-async
Expand All @@ -23,7 +25,7 @@ install and run through flake8
install and run with pre-commit
===============================

If you use `pre-commit <https://pre-commit.com/>`_, you can use it with flake8-async by
If you use `pre-commit <https://pre-commit.com/>`__, you can use it with flake8-async by
adding the following to your ``.pre-commit-config.yaml``:

.. code-block:: yaml
Expand All @@ -38,17 +40,21 @@ adding the following to your ``.pre-commit-config.yaml``:
This is often considerably faster for large projects, because ``pre-commit``
can avoid running ``flake8-async`` on unchanged files.
``flake8-async`` does not retain any memory between files, they are parsed completely independently.

Afterwards, run

.. code-block:: sh
pip install pre-commit flake8-async
pre-commit run .
.. _run_standalone:

install and run as standalone
=============================

If inside a git repository, running without arguments will run it against all ``*.py`` files in the repository.
If inside a git repository, running without arguments will run it against all ``*.py`` files in the repository tree.

.. code-block:: sh
Expand Down Expand Up @@ -81,7 +87,7 @@ Run through ruff
================
`Ruff <https://github.com/astral-sh/ruff>`_ is a linter and formatter that reimplements a lot of rules from various flake8 plugins.

They currently only support a small subset of the rules though, see https://github.com/astral-sh/ruff/issues/8451 for current status and https://docs.astral.sh/ruff/rules/#flake8-async-async for documentation.
They currently only support a small subset of the ``flake8-async`` rules though, see https://github.com/astral-sh/ruff/issues/8451 for current status and https://docs.astral.sh/ruff/rules/#flake8-async-async for documentation.

*************
Configuration
Expand All @@ -105,60 +111,110 @@ config files since flake8>=6, as flake8 tries to validate correct
configuration with a regex. We have decided not to conform to this, as it
would be a breaking change for end-users requiring them to update ``noqa``\ s
and configurations, we think the ``ASYNC`` code is much more readable than
e.g. ``ASYxxx``, and ruff does not enforce such a limit. The easiest option
for users hitting this error is to instead use the ``--disable`` option as
documented `below <#--disable>`__. See further discussion and other
e.g. ``ASYxxx``, and ruff does not enforce such a limit.
The easiest option for users hitting this error is to instead use the :ref:`disable`
option.
See further discussion and other
workarounds in https://github.com/python-trio/flake8-async/issues/230.

.. _enable:

``--enable``
``enable``
------------

Comma-separated list of error codes to enable, similar to flake8 --select but is additionally more performant as it will disable non-enabled visitors from running instead of just silencing their errors.
Comma-separated list of error codes to enable, similar to flake8 --select but is additionally more performant as it will disable non-enabled visitors from running instead of just silencing their errors. Defaults to "ASYNC".

Example
^^^^^^^
.. code-block:: none
.. _--disable:
enable=ASYNC1,ASYNC200
``--disable``
.. _disable:

``disable``
-------------

Comma-separated list of error codes to disable, similar to flake8 ``--ignore`` but is additionally more performant as it will disable non-enabled visitors from running instead of just silencing their errors. It will also bypass errors introduced in flake8>=6, see above.
This is parsed after :ref:`enable`, so if a rule is both "enabled" and "disabled" it will be disabled.
Defaults to "ASYNC9".

Example
^^^^^^^
.. code-block:: none
disable=ASYNC91,ASYNC117
``--autofix``
.. _autofix:

``autofix``
-------------

Comma-separated list of error-codes to enable autofixing for if implemented. Requires running as a standalone program. Pass ``--autofix=ASYNC`` to enable all autofixes.
Comma-separated list of error-codes to enable autofixing for if implemented.
Requires :ref:`running as a standalone program <run_standalone>`.
Only a subset of rules support autofixing, see :ref:`this list <autofix-support>`.
Pass ``--autofix=ASYNC`` to enable all available autofixes.

Defaults to an empty list.

Example
^^^^^^^
.. code-block:: none
``--error-on-autofix``
autofix=ASYNC
``error-on-autofix``
----------------------

Whether to also print an error message for autofixed errors.
Whether to also print an error message for autofixed errors. Defaults to ``False``

Example
^^^^^^^
.. code-block:: none
error-on-autofix=True
Modifying rule behaviour
========================

.. _--anyio:
.. _anyio:

``--anyio``
``anyio``
-----------

Change the default library to be anyio instead of trio. If trio is imported it will assume both are available and print suggestions with [anyio/trio].
Change the default library to be anyio instead of trio. This is mostly used for the sake of printing suggestions in error messages, but may affect some logic. If additional libraries are imported other than the default then rules will assume multiple are available simultaneously. It is currently not possible to set multiple default libraries, other than `anyio`+`asyncio`.

Example
^^^^^^^
.. code-block:: none
``--asyncio``
anyio=True
.. _asyncio:

``asyncio``
-------------
Set default library to be ``asyncio``. See :ref:`--anyio`
Set default library to be ``asyncio``. See :ref:`anyio`

Example
^^^^^^^
.. code-block:: none
asyncio=True
``no-checkpoint-warning-decorators``
------------------------------------

Comma-separated list of decorators to disable checkpointing checks for, turning off ASYNC910 and ASYNC911 warnings for functions decorated with any decorator matching any in the list. Matching is done with `fnmatch <https://docs.python.org/3/library/fnmatch.html>`_. Defaults to disabling for ``asynccontextmanager``.
Comma-separated list of decorators to disable checkpointing checks for, turning off :ref:`ASYNC910 <async910>` and :ref:`ASYNC911 <async911>` warnings for functions decorated with any decorator matching any in the list. Matching is done with `fnmatch <https://docs.python.org/3/library/fnmatch.html>`_. Defaults to disabling for ``asynccontextmanager``.

Decorators-to-match must be identifiers or dotted names only (not PEP-614 expressions), and will match against the name only - e.g. ``foo.bar`` matches ``foo.bar``, ``foo.bar()``, and ``foo.bar(args, here)``, etc.

For example:
Example
^^^^^^^

::
.. code-block:: none
no-checkpoint-warning-decorators =
mydecorator,
Expand All @@ -171,9 +227,13 @@ For example:

Comma-separated list of methods which should be used with ``.start()`` when opening a context manager,
in addition to the default ``trio.run_process``, ``trio.serve_tcp``, ``trio.serve_ssl_over_tcp``, and
``trio.serve_listeners``. Names must be valid identifiers as per ``str.isidentifier()``. For example:
``trio.serve_listeners``. Names must be valid identifiers as per ``str.isidentifier()``.
Used by :ref:`ASYNC113 <async113>`, and :ref:`ASYNC114 <async114>` will warn when encountering methods not in the list.

Example
^^^^^^^

::
.. code-block:: none
startable-in-context-manager =
myfun,
Expand All @@ -182,15 +242,16 @@ in addition to the default ``trio.run_process``, ``trio.serve_tcp``, ``trio.serv
.. _async200-blocking-calls:

``async200-blocking-calls``
---------------------------
-----------------------------

Comma-separated list of pairs of values separated by ``->`` (optional whitespace stripped), where the first is a pattern for a call that should raise an error if found inside an async function, and the second is what should be suggested to use instead. It uses fnmatch as per `no-checkpoint-warning-decorators`_ for matching. The part after ``->`` is not used by the checker other than when printing the error, so you could add extra info there if you want.
Comma-separated list of pairs of values separated by ``->`` (optional whitespace stripped), where the first is a pattern for a call that should raise :ref:`ASYNC200 <async200>` if found inside an async function, and the second is what should be suggested to use instead. It uses fnmatch as per `no-checkpoint-warning-decorators`_ for matching. The part after ``->`` is not used by the checker other than when printing the error, so you could add extra info there if you want.

The format of the error message is ``User-configured blocking sync call {0} in async function, consider replacing with {1}.``, where ``{0}`` is the pattern the call matches and ``{1}`` is the suggested replacement.

Example:
Example
^^^^^^^

::
.. code-block:: none
async200-blocking-calls =
my_blocking_call -> async.alternative,
Expand All @@ -201,7 +262,7 @@ Example:
Specified patterns must not have parentheses, and will only match when the pattern is the name of a call, so given the above configuration

::
.. code-block:: python
async def my_function():
my_blocking_call() # this would raise an error
Expand Down
38 changes: 32 additions & 6 deletions flake8_async/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import subprocess
import sys
import tokenize
import warnings
from argparse import ArgumentParser, ArgumentTypeError, Namespace
from typing import TYPE_CHECKING

Expand All @@ -37,7 +38,7 @@


# CalVer: YY.month.patch, e.g. first release of July 2022 == "22.7.1"
__version__ = "24.5.2"
__version__ = "24.5.3"


# taken from https://github.com/Zac-HD/shed
Expand Down Expand Up @@ -262,7 +263,7 @@ def add_options(option_manager: OptionManager | ArgumentParser):
)
add_argument(
"--startable-in-context-manager",
type=parse_trio114_identifiers,
type=parse_async114_identifiers,
default="",
required=False,
help=(
Expand All @@ -276,7 +277,18 @@ def add_options(option_manager: OptionManager | ArgumentParser):
)
add_argument(
"--trio200-blocking-calls",
type=parse_trio200_dict,
type=parse_async200_dict,
default={},
required=False,
help=(
"Comma-separated list of key->value pairs, where key is a [dotted] "
"function that if found inside an async function will raise ASYNC200, "
"suggesting it be replaced with {value}"
),
)
add_argument(
"--async200-blocking-calls",
type=parse_async200_dict,
default={},
required=False,
help=(
Expand Down Expand Up @@ -348,13 +360,27 @@ def get_matching_codes(
code for code in options.enable if not error_has_subidentifier(code)
)

# we do not use DeprecationWarning, since that is silenced when run as standalone
# or should maybe just print on stderr
if options.trio200_blocking_calls:
warnings.warn(
"trio200-blocking-calls has been deprecated in favor "
"of async200-blocking-calls",
stacklevel=1,
)
assert not options.async200_blocking_calls, (
"You cannot specify both trio200-blocking-calls and "
"async200-blocking-calls. You should only use the latter."
)
options.async200_blocking_calls = options.trio200_blocking_calls

Plugin._options = Options(
enabled_codes=enabled_codes,
autofix_codes=autofix_codes,
error_on_autofix=options.error_on_autofix,
no_checkpoint_warning_decorators=options.no_checkpoint_warning_decorators,
startable_in_context_manager=options.startable_in_context_manager,
trio200_blocking_calls=options.trio200_blocking_calls,
async200_blocking_calls=options.async200_blocking_calls,
anyio=options.anyio,
asyncio=options.asyncio,
disable_noqa=options.disable_noqa,
Expand All @@ -365,15 +391,15 @@ def comma_separated_list(raw_value: str) -> list[str]:
return [s.strip() for s in raw_value.split(",") if s.strip()]


def parse_trio114_identifiers(raw_value: str) -> list[str]:
def parse_async114_identifiers(raw_value: str) -> list[str]:
values = comma_separated_list(raw_value)
for value in values:
if keyword.iskeyword(value) or not value.isidentifier():
raise ArgumentTypeError(f"{value!r} is not a valid method identifier")
return values


def parse_trio200_dict(raw_value: str) -> dict[str, str]:
def parse_async200_dict(raw_value: str) -> dict[str, str]:
res: dict[str, str] = {}
splitter = "->" # avoid ":" because it's part of .ini file syntax
values = [s.strip() for s in raw_value.split(",") if s.strip()]
Expand Down
2 changes: 1 addition & 1 deletion flake8_async/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ class Options:
error_on_autofix: bool
no_checkpoint_warning_decorators: Collection[str]
startable_in_context_manager: Collection[str]
trio200_blocking_calls: dict[str, str]
async200_blocking_calls: dict[str, str]
anyio: bool
asyncio: bool
disable_noqa: bool
Expand Down
Loading

0 comments on commit 9653fde

Please sign in to comment.