Split requirements files

Update CI and other files that refer to them. While we're at it, switch to choco pkg winflexbison3 from winflexbison. Update RELEASE file and template as well, to eliminate possible confusion (not that any has been reported). Signed-off-by: Mats Wichmann <[email protected]>
SCons · Aug 4, 2022 · 6cff654 · 6cff654
1 parent 4aaa505
commit 6cff654
Show file tree

Hide file tree

Showing 10 changed files with 117 additions and 55 deletions.
diff --git a/.appveyor/install.bat b/.appveyor/install.bat
@@ -5,8 +5,8 @@ set PATH=C:\\%WINPYTHON%;C:\\%WINPYTHON%\\Scripts;C:\\ProgramData\\chocolatey\\b
 C:\\%WINPYTHON%\\python.exe -m pip install -U --progress-bar off pip setuptools wheel
 C:\\%WINPYTHON%\\python.exe -m pip install -U --progress-bar off coverage codecov
 set STATIC_DEPS=true & C:\\%WINPYTHON%\\python.exe -m pip install -U --progress-bar off lxml
-C:\\%WINPYTHON%\\python.exe -m pip install -U --progress-bar off -r requirements.txt
+C:\\%WINPYTHON%\\python.exe -m pip install -U --progress-bar off -r requirements-dev.txt
 REM install 3rd party tools to test with
-choco install --allow-empty-checksums dmd ldc swig vswhere xsltproc winflexbison
+choco install --allow-empty-checksums dmd ldc swig vswhere xsltproc winflexbison3
 set SCONS_CACHE_MSVC_CONFIG=true
 set
diff --git a/.github/workflows/scons-package.yml b/.github/workflows/scons-package.yml
@@ -23,10 +23,10 @@ jobs:
       run: |
         python -m pip install --upgrade pip setuptools wheel
         #python -m pip install flake8 pytest
-        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+        if [ -f requirements-pkg.txt ]; then pip install -r requirements-pkg.txt; elif [ -f requirements.txt ]; then pip install -r requirements.txt; else; fi
         sudo apt-get update
         sudo apt-get -y install docbook-xml docbook-xsl xsltproc fop docbook-xsl-doc-pdf
-        # try to keeo the texlive install as small as we can to save some time/space
+        # try to keep the texlive install as small as we can to save some time/space
         sudo apt-get -y --no-install-recommends install texlive biber texmaker ghostscript texlive-latex-base texlive-latex-extra texlive-bibtex-extra texlive-font-utils latexmk
     # This is disabled until the run can be configured to only
     # check the code that matters, else we fail on non-essentials

diff --git a/.travis.yml b/.travis.yml
@@ -15,7 +15,7 @@ os:
 
 install:
   # needed for Docbook tests, must be in virtualenv context
-  - pip install -r requirements.txt
+  - pip install -r requirements-dev.txt
   # do the rest of the image setup
   - ./.travis/install.sh
 

diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
@@ -2,7 +2,7 @@ Contributing to SCons
 #####################
 
 Introduction
-===========
+============
 
 Thanks for taking the time to contribute to SCons!
 
@@ -106,7 +106,7 @@ Documentation is written in a markup language which is a
 light extension of Docbook-XML, and the doc build consists
 of translating to pure docbook, then using standard tools to
 generate HTML and PDF outputs from that. There's lots more
-on the documentation process at the Documentation Toolchain page::
+on the documentation process at the Documentation Toolchain page:
 
     https://github.com/SCons/scons/blob/master/doc/overview.rst
 
@@ -132,7 +132,7 @@ can do that with the ``C`` (change directory) option::
 
 There is another approach that kind of reverses that order:
 construct a Python virtualenv and install the development tree in it.
-If you're not familiar with virtualenvs, there's an example here::
+If you're not familiar with virtualenvs, there's an example here:
 
     https://scons-cookbook.readthedocs.io/en/latest/#setting-up-a-python-virtualenv-for-scons
 
@@ -172,23 +172,38 @@ of Python. The tools which will be used to to actually construct the
 project, such as compilers, documentation production tools, etc.
 should of course be installed by the appropriate means.  In order
 to develop SCons and run its test suite, there are some dependencies,
-listed in the ``requirements.txt`` file.
+listed in the ``requirements-dev.txt`` file. Install these with::
+
+    $ python -m pip install -r requirements-dev.txt
+
+For building the SCons packages and documentation there are some further
+requirements, you can get these with::
+
+    $ python -m pip install -r requirements-pkg.txt
+
+The requirements are inclusive so you only need the latter to get
+everything installed.
+
+There are other, non-Python requirements to do a doc build. These
+are system-specific. See bin/scons_dev_master.py for the set up that
+works for Ubuntu systems.
+
 
 Making Changes
 ==============
 
 Virtually all of the SCons functionality exists in the "build engine," the
 ``SCons`` subdirectory hierarchy that contains all of the modules that
 make up SCons.  The ``scripts/scons.py`` wrapper script exists mainly to find
-the appropriate build engine library and then execute it.
+the appropriate build engine module and execute it.
 
 In order to make your own changes locally and test them by hand, simply edit
 modules in the local ``SCons`` subdirectory tree and then run
-(see the section above about `Executing SCons Without Installing`_)::
+(see the section `Executing SCons Without Installing`_)::
 
     $ python scripts/scons.py [arguments]
 
-(or, if using the virtualenv/editable approach, ``scons [arguents]``)
+Or, if using the virtualenv/editable approach: ``scons [arguments]``
 
 Note that the regular SCons development process makes heavy use of automated
 testing.  See the `Testing`_ and `Typical Development Workflow`_ sections below for more
@@ -320,7 +335,8 @@ subdirectory, and do not require that a build be performed first.
 Typical Development Workflow
 ============================
 
-    Caveat: The point of this section isn't to describe one dogmatic workflow.
+.. hint::
+    The point of this section is not to describe one dogmatic workflow.
     Just running the test suite can be time-consuming, and getting a patch to
     pass all of the tests can be more so.  If you're genuinely blocked, it may
     make more sense to submit a patch with a note about which tests still
@@ -357,7 +373,9 @@ platform, Windows users can translate as appropriate)):
 
       $ python runtest.py -a -o test.log
 
-  Be patient, there are more than 1100 test scripts in the whole suite!
+  Be patient, there are more than 1100 test scripts in the whole suite
+  (using a ``-j`` option pretty much always helps. For example, if you have
+  an 8-core processor, try ``runtest.py -j 8```).
 
   If any test scripts fail, they will be listed in a summary at the end of the
   log file.  Some test scripts may also report NO RESULT because (for example)
@@ -392,11 +410,21 @@ example, if you're making a change to just the Java support, you might start
 looking for regressions by just running the ``test/Java/\*.py`` tests instead of
 running all tests with ``runtest.py -a``.
 
+- To actually submit the fix and any test work as a Pull Request,
+  there will be some version control steps. For example::
+
+      $ git checkout -b fix-1387
+      $ git modified     # check that this reports your expected list
+      $ git add `git modified`
+      $ git commit -s    # fill in a good description of your changes
+
+  And proceed to push the change as a PR.
+
 
 Building Packages
 =================
 
-We use SCons (version 3.1.2 or later) to build its own packages.  If you
+We use SCons (since version 3.1.2) to build its own packages.  If you
 already have an appropriate version of SCons installed on your system,
 you can build everything by simply running it::
 
@@ -411,12 +439,12 @@ you can run the build directly from the source tree
 Those are full builds: depending on the utilities installed on your system,
 any or all of the following packages will be built::
 
-    SCons-4.3.0-py3-none-any.whl
-    SCons-4.3.0ayyyymmdd.tar.gz
-    SCons-4.3.0ayyyymmdd.zip
-    scons-doc-4.3.0ayyyymmdd.tar.gz
-    scons-local-4.3.0ayyyymmdd.tar.gz
-    scons-local-4.3.0ayyyymmdd.zip
+    SCons-4.4.0-py3-none-any.whl
+    SCons-4.4.0ayyyymmdd.tar.gz
+    SCons-4.4.0ayyyymmdd.zip
+    scons-doc-4.4.0ayyyymmdd.tar.gz
+    scons-local-4.4.0ayyyymmdd.tar.gz
+    scons-local-4.4.0ayyyymmdd.zip
 
 The ``SConstruct`` file is supposed to be smart enough to avoid trying to build
 packages for which you don't have the proper utilities installed.
@@ -465,14 +493,14 @@ bootstrap.py
     Obsolete packaging logic - ignore this.
 
 debian/
-    Files needed to construct a Debian package. The contents of this directory
-    are dictated by the
+    Files needed to construct a Debian package.
+    The contents of this directory are dictated by the
     `Debian Policy Manual <https://www.debian.org/doc/debian-policy>`).
     The package will not be accepted into the Debian distribution unless
     the contents of this directory satisfy the relevant Debian policies.
-    At this point, this is a sample; SCons is packaged in the Debian
-    project (and thus inherited by projects which derive from it, if
-    they haven't made their own packages). See:
+    At this point, this is a sample; SCons is packaged for Debian by the
+    Debian project itself (and thus inherited by projects which derive from it,
+    if they haven't made their own packages). See:
 
     - `Debian scons packages <https://packages.debian.org/search?keywords=scons&searchon=names&suite=all&section=all>`_
     - `Ubuntu scons packages <https://packages.ubuntu.com/search?keywords=scons&searchon=names&suite=all&section=all>`_
@@ -528,8 +556,8 @@ SCons/
     documentation stubs kept together with pieces of the engine.
 
 test/
-    End-to-end tests of the SCons utility itself.  These are separate from the
-    individual module unit tests.
+    End-to-end tests of the SCons utility itself.
+    These are separate from the individual module unit tests.
 
 testing/
     SCons testing framework.
diff --git a/RELEASE.txt b/RELEASE.txt
@@ -1,10 +1,17 @@
-A new SCons release, 4.4.1, is now available
-on the SCons download page:
+If you are reading this in the git repository, the contents
+refer to *unreleased* changes since the last SCons release.
+Past official release announcements appear at:
 
-      https://scons.org/pages/download.html
+    https://scons.org/tag/releases.html
 
+==================================================================
 
-Here is a summary of the changes since 4.4.1:
+A new SCons release, 4.4.1, is now available on the SCons download page:
+
+    https://scons.org/pages/download.html
+
+
+Here is a summary of the changes since 4.4.0:
 
 NEW FUNCTIONALITY
 -----------------

diff --git a/pyproject.toml b/pyproject.toml
@@ -0,0 +1,13 @@
+[build-system]
+build-backend = "setuptools.build_meta"
+requires = ["setuptools", "wheel"]
+
+# for black and mypy, set the lowest Python version supported
+[tool.black]
+quiet = true
+target-version = ['py36']
+skip-string-normalization = true
+
+[mypy]
+python_version = 3.6
+
diff --git a/requirements-dev.txt b/requirements-dev.txt
@@ -0,0 +1,12 @@
+# Dependencies for development - mainly for running tests
+
+-r requirements.txt
+
+# for now keep pinning "known working" lxml,
+# it's been a troublesome component in the past.
+lxml==4.9.1
+ninja
+
+# Needed for test/Parallel/failed-build/failed-build.py
+# Also for test/ninja/shutdown_scons_daemon.py
+psutil
diff --git a/requirements-pkg.txt b/requirements-pkg.txt
@@ -0,0 +1,13 @@
+# Dependencies for packaging, and distribution, including the docs build
+
+-r requirements-dev.txt
+
+# Check if the README.rst will render o.k. on pypi and not block an upload
+# Can be used with twinecheck
+# See: https://github.com/pypa/readme_renderer
+readme-renderer
+
+# sphinx pinned because it has broken several times on new releases
+sphinx>=5.1.1
+sphinx_rtd_theme
+rst2pdf
diff --git a/requirements.txt b/requirements.txt
@@ -1,19 +1 @@
-# Packages needed for development, packaging, and distribution, but not for running SCons
-
-# This will check if the README.rst will render o.k. on pypi and not block an upload
-# Can be used with twinecheck
-# See: https://github.com/pypa/readme_renderer
-readme-renderer
-#sphinx<=5.0.0
-sphinx>=5.1.1
-sphinx_rtd_theme
-rst2pdf
-# for now keep pinning "known working" lxml,
-# it's been a troublesome component in the past.
-lxml==4.9.1
-rst2pdf
-ninja
-
-# Needed for test/Parallel/failed-build/failed-build.py
-# Also for test/ninja/shutdown_scons_daemon.py
-psutil
+# No dependencies for running SCons
diff --git a/template/RELEASE.txt b/template/RELEASE.txt
@@ -1,10 +1,17 @@
-A new SCons release, 4.1.0, is now available
-on the SCons download page:
+If you are reading this in the git repository, the contents
+refer to *unreleased* changes since the last SCons release.
+Past official release announcements appear at:
 
-      https://scons.org/pages/download.html
+    https://scons.org/tag/releases.html
 
+==================================================================
 
-Here is a summary of the changes since 4.1.0:
+A new SCons release, 4.4.1, is now available on the SCons download page:
+
+    https://scons.org/pages/download.html
+
+
+Here is a summary of the changes since 4.4.0:
 
 NEW FUNCTIONALITY
 -----------------