From c8a9368a21f64362dc471e5ba2acbe71045e4e31 Mon Sep 17 00:00:00 2001 From: Vytautas Liuolia Date: Thu, 7 Nov 2024 19:54:06 +0100 Subject: [PATCH 1/7] docs(community): create a skeleton for packaging guide --- docs/community/index.rst | 1 + docs/community/packaging.rst | 89 ++++++++++++++++++++++++++++++++++++ 2 files changed, 90 insertions(+) create mode 100644 docs/community/packaging.rst diff --git a/docs/community/index.rst b/docs/community/index.rst index 7e800a910..93e82a5b9 100644 --- a/docs/community/index.rst +++ b/docs/community/index.rst @@ -6,4 +6,5 @@ Community Guide help contributing + packaging ../user/faq diff --git a/docs/community/packaging.rst b/docs/community/packaging.rst new file mode 100644 index 000000000..a07f2888c --- /dev/null +++ b/docs/community/packaging.rst @@ -0,0 +1,89 @@ +Packaging Guide +=============== + +Normally, the recommended way to :ref:`install Falcon ` in your +project is using ``pip`` (or a compatible package manager, such as +``poetry``, ``uv``, and many others) that fetches package archives from +`PyPI`_. + +WiP -- explain why in certain scenarios it is desirable to use other packages... + +This guide is primarily aimed at packagers that create and maintain Falcon +packages for operating systems such as Linux and BSD distributions, as well as +alternative Python distribution channels such as +`conda-forge `__. + + +Obtaining Release +----------------- + +In order to package a specific Falcon release, you first need to obtain its +source files. + +It is up to you which authoritative source to use. +The most common alternatives are: + +* Source distribution (aka "*sdist*") on PyPI. + +* GitHub release archive. + +* Clone GitHub repository. + + +Building Binaries +----------------- + +WiP... + + +Building Documentation +---------------------- + +It is quite uncommon to also include offline documentation (or to provide a +separate documentation package) as the user can simply browse our documentation +at `Read the Docs `__. Even if the package does +not contain the latest version of Falcon, it is possible to switch to an +older one using Read the Docs version picker. + +If you do decide to ship the offline docs too, you can build it using +``docs/Makefile`` (you can also invoke ``sphinx-build`` directly): + +* To build HTML docs, use ``make html``. + + The resulting files will be built in ``docs/_build/html/``. + +* To build man pages, use ``make man``. + + The resulting man page file will be called ``docs/_build/man/falcon.1``. + You can check it manually using ``man``:: + + man docs/_build/man/falcon.1 + + You will need to rename this file to match your package naming standards, and + copy it an appropriate man page directory + (typically under ``/usr/share/man/`` or similar). + + +Testing Package +--------------- + +WiP... + +.. tip:: + You can run ``pytest`` against Falcon's ``tests/`` from any directory, + i.e., the below should work just fine:: + + /usr/local/foo-bin/pytest /bar/baz/falcon-release-dir/tests + + This pattern is regularly exercised in our Continuous Integration gates, + as ``cibuildwheel`` (see above) does not run tests from the project's + directory either. + + +Thank You +--------- + +WiP... + + +.. _PyPI: https://pypi.org/project/falcon/ From 1cce4655e8b170fbdfb15ecd30109a5428f61c09 Mon Sep 17 00:00:00 2001 From: Vytautas Liuolia Date: Sat, 9 Nov 2024 21:36:05 +0100 Subject: [PATCH 2/7] docs(community): further improve pkg guide --- docs/community/packaging.rst | 85 ++++++++++++++++++++++++++++++++++-- 1 file changed, 81 insertions(+), 4 deletions(-) diff --git a/docs/community/packaging.rst b/docs/community/packaging.rst index a07f2888c..03758b527 100644 --- a/docs/community/packaging.rst +++ b/docs/community/packaging.rst @@ -13,6 +13,9 @@ packages for operating systems such as Linux and BSD distributions, as well as alternative Python distribution channels such as `conda-forge `__. +.. note:: + Unless noted otherwise in specific sections, this document is only + applicable to Falcon 4.0.1 or newer. Obtaining Release ----------------- @@ -30,10 +33,84 @@ The most common alternatives are: * Clone GitHub repository. +Metadata and Dependencies +------------------------- + +It is recommend to synchronize the metadata such as the project's description +with recent releases on `PyPI`_. + +Falcon has **no hard runtime dependencies** except the standard Python +library. So depending on how Python is packaged in your distribution +(i.e., whether parts of the stdlib are potentially broken out to separate +packages), Falcon should only depend on the basic installation of Python. + +.. note:: + Falcon has no third-party dependencies since 2.0, however, we were + vendoring the ``python-mimeparse`` library (which also had a different + licence, MIT versus Falcon's Apache 2.0). + + This is no longer a concern as the relevant functionality has been + reimplemented from scratch in Falcon 4.0, also fixing some long standing + behavioral quirks and bugs on the way. + + +Supported Platforms +------------------- + +Falcon supports a wide range of platforms -- the pure-Python version should be +able to run anywhere a supported version of Python runs. + + Building Binaries ----------------- -WiP... +The absolute minimum in terms of packaging is not building any binaries, but +just distributing the Python modules found under ``falcon/``. This is roughly +equivalent to our pure-Python wheel on `PyPI`_. + +The framework would still function just fine, however, the overall performance +would be somewhat (~30-40%) lower, and potentially much lower (an order of +magnitude) for certain "hot" code paths that feature a dedicated implementation +in Cython. + +.. note:: + The above notes on performance only apply to CPython. + + In the unlikely case you are packaging Falcon for PyPy, we recommend simply + sticking to pure-Python code. + +In order to build a binary package, you will obviously need a compiler +toolchain, and the CPython library headers. +Hopefully your distribution already has Python tooling that speaks +`PEP 517 `__ -- this is how the framework's +build process is implemented +(using the popular `setuptools `__). + +We also use `cibuildwheel`_ to build our packages that are later uploaded to +`PyPI`_, but we realize that its isolated, Docker-centric approach probably +does not lend itself very well to packaging for a distribution of an operating +system. + +If your build process does not support installation of build dependencies in +a PEP 517 compatible way, you will also have to install Cython yourself +(version 3.0.8 or newer is recommended to build Falcon). + +Big-endian support +^^^^^^^^^^^^^^^^^^ +We regularly build and test :ref:`binary wheels ` on the +IBM Z platform (aka ``s390x``) which is big-endian. +We are not aware of endianness-related issues. + +32-bit support +^^^^^^^^^^^^^^ +Falcon is not very well tested on 32-bit systems, and we do not provide any +32-bit binary wheels either. We even explicitly fall back to pure-Python code +in some cases such as the multipart form parser (as the smaller ``Py_ssize_t`` +would interfere with uploading of files larger than 2 GiB) if we detect a +32-bit flavor of CPython. + +If you do opt to provide 32-bit binary Falcon packages, make sure that your run +:ref:`extensive tests ` against the built package. Building Documentation @@ -55,15 +132,14 @@ If you do decide to ship the offline docs too, you can build it using * To build man pages, use ``make man``. The resulting man page file will be called ``docs/_build/man/falcon.1``. - You can check it manually using ``man``:: - - man docs/_build/man/falcon.1 You will need to rename this file to match your package naming standards, and copy it an appropriate man page directory (typically under ``/usr/share/man/`` or similar). +.. _packaging_testing: + Testing Package --------------- @@ -87,3 +163,4 @@ WiP... .. _PyPI: https://pypi.org/project/falcon/ +.. _cibuildwheel: https://cibuildwheel.pypa.io/ From 759a101236700ca5492390cc8d7d7c88dcf2cb29 Mon Sep 17 00:00:00 2001 From: Vytautas Liuolia Date: Thu, 14 Nov 2024 19:10:25 +0100 Subject: [PATCH 3/7] docs(community): flesh out pkg guide --- docs/community/packaging.rst | 81 ++++++++++++++++++++++++++++-------- 1 file changed, 63 insertions(+), 18 deletions(-) diff --git a/docs/community/packaging.rst b/docs/community/packaging.rst index 03758b527..5a243aa35 100644 --- a/docs/community/packaging.rst +++ b/docs/community/packaging.rst @@ -1,14 +1,19 @@ Packaging Guide =============== -Normally, the recommended way to :ref:`install Falcon ` in your -project is using ``pip`` (or a compatible package manager, such as +Normally, the recommended way to :ref:`install Falcon ` into your +project is using ``pip`` (or a compatible package/project manager, such as ``poetry``, ``uv``, and many others) that fetches package archives from `PyPI`_. -WiP -- explain why in certain scenarios it is desirable to use other packages... +However, the `PyPI`_-based way of installation is not always applicable or +optimal. For instance, when the system package manager of a Linux distribution +is used to install Python-based software, it normally gets the dependencies +from the same distribution channel, as the specific versions of the packages +(often not without minor patches) were carefully tested to work well together +on the operating system in question. -This guide is primarily aimed at packagers that create and maintain Falcon +This guide is primarily aimed at engineers who create and maintain Falcon packages for operating systems such as Linux and BSD distributions, as well as alternative Python distribution channels such as `conda-forge `__. @@ -17,20 +22,58 @@ alternative Python distribution channels such as Unless noted otherwise in specific sections, this document is only applicable to Falcon 4.0.1 or newer. +If you run into any packaging issues, questions that this guide does not cover, +or just find a bug in Falcon, please :ref:`let us know `! + + Obtaining Release ----------------- In order to package a specific Falcon release, you first need to obtain its -source files. +source archive. It is up to you which authoritative source to use. The most common alternatives are: -* Source distribution (aka "*sdist*") on PyPI. +* **Source distribution** (aka "*sdist*") on `PyPI`_. + + If you are unsure, the recommended way is to use our source distribution from + `PyPI`_ (also available on GitHub releases, see below). + + You can query PyPA Warehouse's + `JSON API `__ in order to + obtain the latest stable version of Falcon, fetch the *sdist* URL, and more. + + The API URL specifically for Falcon is https://pypi.org/pypi/falcon/json. + Here is how you can query it using the popular ``requests``: + + >>> import requests + >>> resp = requests.get('https://pypi.org/pypi/falcon/json') + >>> for url in resp.json()['urls']: + ... if url['packagetype'] == 'sdist': + ... print(f'Latest Falcon sdist: {url["url"]}') + ... + Latest Falcon sdist: https://files.pythonhosted.org/<...>/falcon-4.0.2.tar.gz + + (``4.0.2`` was the latest version at the time of this writing.) + +* **GitHub release archive**. -* GitHub release archive. + Alternatively, you can download the archive from our + `Releases on GitHub `__. + GitHub automatically archives the whole repository for every release, and + attaches the tarball to the release page. In addition, our release automation + also uploads the *sdist* (see above) to the release as well. -* Clone GitHub repository. +* **Clone GitHub repository**. + + If your packaging workflow is based on a Git repository that tracks both the + framework's source code, and your patches or tooling scripts, you will + probably want to clone our + `GitHGub repository `__ instead. + + Every release has a corresponding annotated Git tag that shares the name + with the package version on PyPI, e.g., ``4.0.2``. Metadata and Dependencies @@ -42,7 +85,8 @@ with recent releases on `PyPI`_. Falcon has **no hard runtime dependencies** except the standard Python library. So depending on how Python is packaged in your distribution (i.e., whether parts of the stdlib are potentially broken out to separate -packages), Falcon should only depend on the basic installation of Python. +packages), Falcon should only depend on the basic installation of the targeted +Python interpreter. .. note:: Falcon has no third-party dependencies since 2.0, however, we were @@ -50,17 +94,10 @@ packages), Falcon should only depend on the basic installation of Python. licence, MIT versus Falcon's Apache 2.0). This is no longer a concern as the relevant functionality has been - reimplemented from scratch in Falcon 4.0, also fixing some long standing + reimplemented from scratch in Falcon 4.0.0, also fixing some long standing behavioral quirks and bugs on the way. -Supported Platforms -------------------- - -Falcon supports a wide range of platforms -- the pure-Python version should be -able to run anywhere a supported version of Python runs. - - Building Binaries ----------------- @@ -156,10 +193,18 @@ WiP... directory either. +.. _packaging_thank_you: + Thank You --------- -WiP... +If you are already maintaining Falcon packages, thank you! + +Although we do not have the bandwidth to maintain Falcon packages for any +distribution channel beyond `PyPI`_ ourselves, we are happy to help if you run +into any problems. File an +`issue on GitHub `__, +or just :ref:`send us a message `! .. _PyPI: https://pypi.org/project/falcon/ From cf789e1189ee6107793be487c1894f1ee54a7d75 Mon Sep 17 00:00:00 2001 From: Vytautas Liuolia Date: Thu, 14 Nov 2024 21:53:13 +0100 Subject: [PATCH 4/7] docs(community): finalize pkg docs --- docs/community/packaging.rst | 68 ++++++++++++++++++++++++++++++------ 1 file changed, 58 insertions(+), 10 deletions(-) diff --git a/docs/community/packaging.rst b/docs/community/packaging.rst index 5a243aa35..b273295b2 100644 --- a/docs/community/packaging.rst +++ b/docs/community/packaging.rst @@ -97,6 +97,21 @@ Python interpreter. reimplemented from scratch in Falcon 4.0.0, also fixing some long standing behavioral quirks and bugs on the way. +Optional dependencies +^^^^^^^^^^^^^^^^^^^^^ +Falcon has no official list of optional dependencies, but if you want to +provide "suggested packages" or similar, various media (de-) serialization +libraries can make good candidates, especially those that have official media +handlers such as ``msgpack`` (:class:`~falcon.media.MessagePackHandler`). +:class:`~falcon.media.JSONHandler` can be easily customized using faster JSON +implementations such as ``orjson``, ``rapidjson``, etc, so you can suggest +those that are already packaged for your distribution. + +Otherwise, various ASGI and WSGI application servers could also fit the bill. + +See also :ref:`packaging_test_deps` for the list of third party libraries that +we test against in our Continuous Integration (CI) tests. + Building Binaries ----------------- @@ -136,7 +151,7 @@ Big-endian support ^^^^^^^^^^^^^^^^^^ We regularly build and test :ref:`binary wheels ` on the IBM Z platform (aka ``s390x``) which is big-endian. -We are not aware of endianness-related issues. +We are not aware of any endianness-related issues. 32-bit support ^^^^^^^^^^^^^^ @@ -146,7 +161,7 @@ in some cases such as the multipart form parser (as the smaller ``Py_ssize_t`` would interfere with uploading of files larger than 2 GiB) if we detect a 32-bit flavor of CPython. -If you do opt to provide 32-bit binary Falcon packages, make sure that your run +If you do opt to provide 32-bit Falcon binaries, make sure that you run :ref:`extensive tests ` against the built package. @@ -180,17 +195,50 @@ If you do decide to ship the offline docs too, you can build it using Testing Package --------------- -WiP... +When your Falcon package is ready, it is a common (highly recommended!) +practice to install it into your distribution, and run tests verifying that the +package functions as intended. -.. tip:: - You can run ``pytest`` against Falcon's ``tests/`` from any directory, - i.e., the below should work just fine:: +As of Falcon 4.0+, the only hard test dependency is ``pytest``. - /usr/local/foo-bin/pytest /bar/baz/falcon-release-dir/tests +You can simply run it against Falcon's test suite found in the ``tests/`` +subdirectory:: - This pattern is regularly exercised in our Continuous Integration gates, - as ``cibuildwheel`` (see above) does not run tests from the project's - directory either. + pytest tests/ + +These tests will provide decent (98-99%), although not complete, code coverage, +and should ensure that the basic wiring of your package is correct +(however, see also the next chapter: :ref:`packaging_test_deps`). + +.. tip:: + You can run ``pytest`` from any directory, i.e., the below should work just + fine:: + + /usr/local/foo-bin/pytest /bar/baz/falcon-release-dir/tests/ + + This pattern is regularly exercised in our CI gates, as `cibuildwheel`_ + (see above) does not run tests from the project's directory either. + +.. _packaging_test_deps: + +Optional test dependencies +^^^^^^^^^^^^^^^^^^^^^^^^^^ +As mentioned above, Falcon has no hard test dependencies except ``pytest``, +however, our test suite includes optional integration tests against a selection +of third-party libraries. + +When building :ref:`wheels ` with `cibuildwheel`_, we install a +small subset of the basic optional test dependencies, see the +``requirements/cibwtest`` file in the repository. +Furthermore, when running our full test suite in the CI, we exercise +integration with a larger number of optional libraries and applications servers +(see the ``requirements/tests`` file, as well as various ASGI/WSGI server +integration test definitions in ``tox.ini``). + +Ideally, if your distribution also provides packages for any of the above +optional test dependencies, it may be a good idea to install them into your +test environment as well. This will help verifying that your Falcon package is +compatible with the specific versions of these packages in your distribution. .. _packaging_thank_you: From 37b762d178e4f892bd4fcdcc83a0b5620f04f88f Mon Sep 17 00:00:00 2001 From: Vytautas Liuolia Date: Sat, 16 Nov 2024 19:38:48 +0100 Subject: [PATCH 5/7] docs(community): address a couple of review comments --- docs/community/packaging.rst | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/docs/community/packaging.rst b/docs/community/packaging.rst index b273295b2..6f6b61b00 100644 --- a/docs/community/packaging.rst +++ b/docs/community/packaging.rst @@ -10,8 +10,8 @@ However, the `PyPI`_-based way of installation is not always applicable or optimal. For instance, when the system package manager of a Linux distribution is used to install Python-based software, it normally gets the dependencies from the same distribution channel, as the specific versions of the packages -(often not without minor patches) were carefully tested to work well together -on the operating system in question. +were carefully tested to work well together on the operating system in +question. This guide is primarily aimed at engineers who create and maintain Falcon packages for operating systems such as Linux and BSD distributions, as well as @@ -175,7 +175,15 @@ not contain the latest version of Falcon, it is possible to switch to an older one using Read the Docs version picker. If you do decide to ship the offline docs too, you can build it using -``docs/Makefile`` (you can also invoke ``sphinx-build`` directly): +``docs/Makefile`` (you can also invoke ``sphinx-build`` directly). + +.. note:: + Building the HTML documentation requires the packages listed in + ``requirements/docs``. + + Building ``man`` pages requires only Sphinx itself and the plugins + referenced directly in ``docs/conf.py`` + (currently ``myst-parser``, ``sphinx-copybutton``, and ``sphinx-design``). * To build HTML docs, use ``make html``. From 23cdca3738d5790cf281592d6bf60fe5064e49e0 Mon Sep 17 00:00:00 2001 From: Vytautas Liuolia Date: Sat, 16 Nov 2024 23:18:45 +0100 Subject: [PATCH 6/7] docs(community/packaging): add a note on SemVer + other improvements --- docs/community/packaging.rst | 33 +++++++++++++++++++++++++++++++++ docs/user/intro.rst | 2 +- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/docs/community/packaging.rst b/docs/community/packaging.rst index 6f6b61b00..06ac539f0 100644 --- a/docs/community/packaging.rst +++ b/docs/community/packaging.rst @@ -76,6 +76,33 @@ The most common alternatives are: with the package version on PyPI, e.g., ``4.0.2``. +Semantic Versioning +------------------- + +Falcon strictly adheres to `SemVer `__ -- incompatible API +changes are only introduced in conjunction with a major version increment. + +When updating your Falcon package, you should always carefully review +:doc:`the changelog ` for the new release that you targeting, +especially if you are moving up to a new SemVer major version. +(In that case, the release notes will include a "Breaking Changes" section.) + +For a packager, another section worth checking is called +"Changes to Supported Platforms", where we announce support for new Python +interpreter versions (or even new implementations), as well as deprecate or +remove the old ones. +While there seems to be +`no clear consensus `__ on whether +removing platform support constitutes a SemVer breaking change, Falcon assumes +that it does (unless we have communicated otherwise in advance, e.g., the +:doc:`Falcon 4.x series ` only guarantee Python 3.10+ support). + +.. note:: + The SemVer guarantees primarily cover the publicly documented API from the + framework user's perspective, so even a minor release may contain important + changes to the build process, tests, and project tooling. + + Metadata and Dependencies ------------------------- @@ -96,6 +123,7 @@ Python interpreter. This is no longer a concern as the relevant functionality has been reimplemented from scratch in Falcon 4.0.0, also fixing some long standing behavioral quirks and bugs on the way. + As a result, the Falcon 4.x series currently has no vendored dependencies. Optional dependencies ^^^^^^^^^^^^^^^^^^^^^ @@ -120,6 +148,11 @@ The absolute minimum in terms of packaging is not building any binaries, but just distributing the Python modules found under ``falcon/``. This is roughly equivalent to our pure-Python wheel on `PyPI`_. +.. tip:: + The easiest way to skip the binaries is to set the + ``FALCON_DISABLE_CYTHON`` environment variable to a non-empty value in the + build environment. + The framework would still function just fine, however, the overall performance would be somewhat (~30-40%) lower, and potentially much lower (an order of magnitude) for certain "hot" code paths that feature a dedicated implementation diff --git a/docs/user/intro.rst b/docs/user/intro.rst index ba35f6199..d7fdb2cd2 100644 --- a/docs/user/intro.rst +++ b/docs/user/intro.rst @@ -55,7 +55,7 @@ Falcon + PyPy first. **Reliable.** We go to great lengths to avoid introducing breaking changes, and when we do they are fully documented and only introduced (in the spirit of -`SemVer `__) with a major version +`SemVer `__) with a major version increment. The code is rigorously tested with numerous inputs and we require 100% coverage at all times. Falcon does not depend on any external Python packages. From 5a454852a323813fdba47828465778a3f3a29a92 Mon Sep 17 00:00:00 2001 From: Vytautas Liuolia Date: Sat, 16 Nov 2024 23:21:43 +0100 Subject: [PATCH 7/7] docs(community): a minor grammar touchup --- docs/community/packaging.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/community/packaging.rst b/docs/community/packaging.rst index 06ac539f0..90b47a224 100644 --- a/docs/community/packaging.rst +++ b/docs/community/packaging.rst @@ -95,7 +95,7 @@ While there seems to be `no clear consensus `__ on whether removing platform support constitutes a SemVer breaking change, Falcon assumes that it does (unless we have communicated otherwise in advance, e.g., the -:doc:`Falcon 4.x series ` only guarantee Python 3.10+ support). +:doc:`Falcon 4.x series ` only guarantees Python 3.10+ support). .. note:: The SemVer guarantees primarily cover the publicly documented API from the