📦 Move packaging to PEP 517 in-tree backend

This essentially allows the cythonization opt-out be controlled by the
`--pure-python` PEP 517 config setting that can be passed to
the corresponding build frontends via their respective CLIs.

.github/workflows/ci-cd.yml

@@ -38,11 +38,9 @@ jobs:
       uses: py-actions/py-dependency-install@v4
       with:
         path: requirements/lint.txt
-    - name: Install itself
+    - name: Self-install
       run: |
-        python setup.py install
-      env:
-        YARL_NO_EXTENSIONS: 1
+        pip install .
     - name: Run linters
       run: |
         make lint
@@ -55,10 +53,8 @@ jobs:
         make doc-spelling
     - name: Prepare twine checker
       run: |
-        pip install -U twine wheel
-        python setup.py sdist bdist_wheel
-      env:
-        YARL_NO_EXTENSIONS: 1
+        pip install -U build twine
+        python -m build --config-settings=--pure-python=true
     - name: Run twine checker
       run: |
         twine check dist/*
@@ -111,23 +107,25 @@ jobs:
       uses: py-actions/py-dependency-install@v4
       with:
         path: requirements/cython.txt
-    - name: Cythonize
-      if: ${{ matrix.no-extensions == '' }}
-      run: |
-        make cythonize
     - name: Install dependencies
       uses: py-actions/py-dependency-install@v4
       with:
         path: requirements/ci.txt
-      env:
-        YARL_NO_EXTENSIONS: ${{ matrix.no-extensions }}
+    - name: Self-install
+      run: >-
+        python -Im pip install -e .
+        --config-settings=--pure-python=${{
+          matrix.no-extensions != ''
+          && 'true'
+          || 'false'
+        }}
     - name: Run unittests
       env:
         COLOR: 'yes'
         YARL_NO_EXTENSIONS: ${{ matrix.no-extensions }}
       run: |
-        python -m pytest tests -vv
-        python -m coverage xml
+        python -Im pytest tests -vv
+        python -Im coverage xml
     - name: Upload coverage
       uses: codecov/[email protected]
       with:
@@ -167,16 +165,12 @@ jobs:
       uses: actions/checkout@v4
     - name: Setup Python
       uses: actions/setup-python@v4
-    - name: Install cython
-      uses: py-actions/py-dependency-install@v4
-      with:
-        path: requirements/cython.txt
-    - name: Cythonize
+    - name: Install pypa/build
       run: |
-        make cythonize
+        python -Im pip install build
     - name: Make sdist
       run:
-        python setup.py sdist
+        python -Im build --sdist
     - name: Upload artifacts
       uses: actions/upload-artifact@v3
       with:
@@ -222,9 +216,6 @@ jobs:
       uses: py-actions/py-dependency-install@v4
       with:
         path: requirements/cython.txt
-    - name: Cythonize
-      run: |
-        make cythonize
     - name: Build wheels
       uses: pypa/[email protected]
       env:

.pre-commit-config.yaml

@@ -7,6 +7,8 @@ repos:
   rev: v1.3.0
   hooks:
   - id: yesqa
+    additional_dependencies:
+    - wemake-python-styleguide
 - repo: https://github.com/PyCQA/isort
   rev: '5.11.5'
   hooks:

CHANGES/893.misc.rst

@@ -0,0 +1,24 @@
+Replaced the packaging is replaced from an old-fashioned :file:`setup.py` to an
+in-tree :pep:`517` build backend -- by :user:`webknjaz`.
+
+Whenever the end-users or downstream packagers need to build ``yarl`` from
+source (a Git checkout or an sdist), they may pass a ``config_settings``
+flag ``--pure-python``. If this flag is not set, a C-extension will be built
+and included into the distribution.
+
+Here is how this can be done with ``pip``:
+
+.. code-block:: console
+
+    $ python -m pip install . --config-settings=--pure-python=
+
+This will also work with ``-e | --editable``.
+
+The same can be achieved via ``pypa/build``:
+
+.. code-block:: console
+
+    $ python -m build --config-setting=--pure-python=
+
+Adding ``-w | --wheel`` can force ``pypa/build`` produce a wheel from source
+directly, as opposed to building an ``sdist`` and then building from it.

MANIFEST.in

@@ -2,6 +2,7 @@ include LICENSE
 include CHANGES.rst
 include README.rst
 graft yarl
+graft packaging
 graft docs
 graft tests
 include yarl/*.c

Makefile

@@ -25,8 +25,8 @@ yarl/%.c: yarl/%.pyx
 cythonize: .cythonize
 
 
-.develop: .install-deps $(shell find yarl -type f) .cythonize
-	@pip install -e .
+.develop: .install-deps $(shell find yarl -type f)
+	@pip install -e . --config-settings=--pure-python=false
 	@touch .develop
 
 fmt:

README.rst

@@ -110,12 +110,13 @@ manylinux-compliant because of the missing glibc and therefore, cannot be
 used with our wheels) the the tarball will be used to compile the library from
 the source code. It requires a C compiler and and Python headers installed.
 
-To skip the compilation you must explicitly opt-in by setting the `YARL_NO_EXTENSIONS`
-environment variable to a non-empty value, e.g.:
+To skip the compilation you must explicitly opt-in by using a PEP 517 config setting
+``--pure-python``, or setting the ``YARL_NO_EXTENSIONS`` environment variable to a
+non-empty value, e.g.:
 
-.. code-block:: bash
+.. code-block:: console
 
-   $ YARL_NO_EXTENSIONS=1 pip install yarl
+   $ pip install yarl --config-settings=--pure-python=
 
 Please note that the pure-Python (uncompiled) version is much slower. However,
 PyPy always uses a pure-Python implementation, and, as such, it is unaffected

docs/index.rst

@@ -101,12 +101,13 @@ manylinux-compliant because of the missing glibc and therefore, cannot be
 used with our wheels) the the tarball will be used to compile the library from
 the source code. It requires a C compiler and and Python headers installed.
 
-To skip the compilation you must explicitly opt-in by setting the ``YARL_NO_EXTENSIONS``
-environment variable to a non-empty value, e.g.:
+To skip the compilation you must explicitly opt-in by using a PEP 517 config setting
+``--pure-python``, or setting the ``YARL_NO_EXTENSIONS`` environment variable to a
+non-empty value, e.g.:
 
-::
+.. code-block:: console
 
-   $ YARL_NO_EXTENSIONS=1 pip install yarl
+   $ pip install yarl --config-settings=--pure-python=
 
 Please note that the pure-Python (uncompiled) version is much slower. However,
 PyPy always uses a pure-Python implementation, and, as such, it is unaffected

packaging/README.md

@@ -0,0 +1,11 @@
+# `pep517_backend` in-tree build backend
+
+The `pep517_backend.hooks` importable exposes callables declared by PEP 517
+and PEP 660 and is integrated into `pyproject.toml`'s
+`[build-system].build-backend` through `[build-system].backend-path`.
+
+# Design considerations
+
+`__init__.py` is to remain empty, leaving `hooks.py` the only entrypoint
+exposing the callables. The logic is contained in private modules. This is
+to prevent import-time side effects.

packaging/pep517_backend/__init__.py

@@ -0,0 +1 @@
+"""PEP 517 build backend for optionally pre-building Cython."""