Pre-commit, darker, blacken-docs, flake8, pylint, bandit (conda#11009)

* Create pyproject.toml

* Create .pre-commit-config.yaml

* Include pre-commit in devenv

* Update CONTRIBUTING.md

.pre-commit-config.yaml

@@ -0,0 +1,39 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.0.1
+    hooks:
+      # standard end of line/end of file cleanup
+      - id: mixed-line-ending
+      - id: end-of-file-fixer
+      - id: trailing-whitespace
+      # ensure syntaxes are valid
+      - id: check-toml
+      - id: check-yaml
+      # catch git merge/rebase problems
+      - id: check-merge-conflict
+  - repo: https://github.com/akaihola/darker
+    rev: 1.3.2
+    hooks:
+      - id: darker
+        additional_dependencies: [black==21.9b0]
+        exclude: ^conda/_vendor/
+  - repo: https://github.com/asottile/blacken-docs
+    rev: v1.11.0
+    hooks:
+      - id: blacken-docs
+        additional_dependencies: [black==21.9b0]
+  - repo: https://github.com/PyCQA/flake8
+    rev: 4.0.1
+    hooks:
+      - id: flake8
+  - repo: https://github.com/PyCQA/pylint
+    rev: v2.11.1
+    hooks:
+      - id: pylint
+        args: [--exit-zero]
+  - repo: https://github.com/PyCQA/bandit
+    rev: 1.7.0
+    hooks:
+      - id: bandit
+        args: [--exit-zero]
+        exclude: ^(conda/_vendor|tests)/

CONTRIBUTING.md

@@ -17,97 +17,168 @@ The conda organization adheres to the [NumFOCUS Code of Conduct](https://www.num
 
 ## Development Environment
 
-### Bash (e.g. macOS, Linux, Windows)
+0. [Signup for a GitHub account][github signup] (if you haven't already) and
+   [install Git on your system][install git].
+1. Fork the conda repository to your personal GitHub account by clicking the
+   "Fork" button on https://github.com/conda/conda and follow GitHub's
+   instructions.
+2. Clone the repo you just forked on GitHub to your local machine. Configure
+   your repo to point to both "upstream" (the main conda repo) and your fork
+   ("origin"). For detailed directions, see below:
 
-0. [Signup for a GitHub account][github signup] (if you haven't already)
-   and [install Git on your system][install git].
+   **Bash (macOS, Linux, Windows)**
 
-1. Fork the conda/conda repository to your personal GitHub account by
-   clicking the "Fork" button on https://github.com/conda/conda and then
-   following the instructions GitHub provides.
+   ```bash
+   # choose the repository location
+   # warning: not the location of an existing conda installation!
+   $ CONDA_PROJECT_ROOT="$HOME/conda"
 
-2. Clone the conda repo you just forked on GitHub to your filesystem anywhere
-   you choose. A special development environment will be set up within the
-   git clone directory below.
-   Set up a new `git remote` to point to both "upstream" (the main conda
-   repo) and your fork repo. For detailed directions, see below.
+   # clone the project
+   # replace `your-username` with your actual GitHub username
+   $ git clone [email protected]:your-username/conda "$CONDA_PROJECT_ROOT"
+   $ cd "$CONDA_PROJECT_ROOT"
 
-   2a. Choose where you want the repository located (not a location of an
-       existing conda installation though!), e.g.:
+   # set the `upstream` as the the main repository
+   $ git remote add upstream [email protected]:conda/conda
+   ```
 
-       CONDA_PROJECT_ROOT="$HOME/conda"
+   **cmd.exe (Windows)**
 
-   2b. Clone the project, with `upstream` being the main repository.
-       Please replace `your-username` with your actual GitHub username.
+   ```batch
+   # choose the repository location
+   # warning: not the location of an existing conda installation!
+   > set "CONDA_PROJECT_ROOT=%HOMEPATH%\conda"
 
-       GITHUB_USERNAME=your-username
-       git clone [email protected]:$GITHUB_USERNAME/conda "$CONDA_PROJECT_ROOT"
-       cd "$CONDA_PROJECT_ROOT"
-       git remote add upstream [email protected]:conda/conda
+   # clone the project
+   # replace `your-username` with your actual GitHub username
+   > git clone [email protected]:your-username/conda "%CONDA_PROJECT_ROOT%"
+   > cd "%CONDA_PROJECT_ROOT%"
 
-3. Create a local development environment, and activate that environment
+   # set the `upstream` as the main repository
+   > git remote add upstream [email protected]:conda/conda
+   ```
 
-       source ./dev/start
+3. Create a local development environment and activate that environment
 
-   This command will create a project-specific base environment at `./devenv`.
-   If the environment already exists, this command will just quickly activate
-   the already-created `./devenv` environment.
+   **Bash (macOS, Linux, Windows)**
+
+   ```bash
+   $ source ./dev/start
+   ```
+
+   **cmd.exe (Windows)**
+
+   ```batch
+   > .\dev\start.bat
+   ```
+
+   This command will create a project-specific base environment (see `devenv`
+   in your repo directory after running this command). If the base environment
+   already exists this command will simply activate the already-created
+   `devenv` environment.
 
    To be sure that the conda code being interpreted is the code in the project
    directory, look at the value of `conda location:` in the output of
    `conda info --all`.
 
-4. Run conda's unit tests using GNU make
+## Static Code Analysis
 
-       make unit
+This project is configured with [pre-commit](https://pre-commit.com/) to
+automatically run linting and other static code analysis on every commit.
+Running these tools prior to the PR/code review process helps in two ways:
 
-   or alternately with pytest
+1. it helps *you* by automating the nitpicky process of identifying and
+   correcting code style/quality issues
+2. it helps *us* where during code review we can focus on the substance of
+   your contribution
 
-       pytest -m "not integration" conda tests
+Feel free to read up on everything pre-commit related in their
+[docs](https://pre-commit.com/#quick-start) but we've included the gist of
+what you need to get started below:
 
-   or you can use pytest to focus on one specific test
+**Bash (macOS, Linux, Windows)**
 
-       pytest tests/test_create.py -k create_install_update_remove_smoketest
+```bash
+# reuse the development environment created above
+$ source ./dev/start
 
-### cmd.exe shell (Windows)
+# install pre-commit hooks for conda
+$ cd "$CONDA_PROJECT_ROOT"
+$ pre-commit install
 
-0. [Signup for a GitHub account][github signup] (if you haven't already)
-   and [install Git on your system][install git].
+# manually running pre-commit on current changes
+# note: by default pre-commit only runs on staged files
+$ pre-commit run
 
-1. Fork the conda/conda repository to your personal GitHub account by
-   clicking the "Fork" button on https://github.com/conda/conda and then
-   following the instructions GitHub provides.
+# automatically running pre-commit during commit
+$ git commit
+```
 
-2. Clone the conda repo you just forked on GitHub to your filesystem anywhere
-   you choose. A special development environment will be set up within the
-   git clone directory below.
-   Set up a new `git remote` to point to both "upstream" (the main conda
-   repo) and your fork repo. For detailed directions, see below.
+**cmd.exe (Windows)**
 
-   2a. Choose where you want the repository located (not a location of an
-       existing conda installation though!), e.g.:
+```batch
+:: reuse the development environment created above
+> .\dev\start.bat
 
-       set "CONDA_PROJECT_ROOT=%HOMEPATH%\conda"
+:: install pre-commit hooks for conda
+> cd "%CONDA_PROJECT_ROOT%"
+> pre-commit install
 
-   2b. Clone the project, with `upstream` being the main repository.
-       Please replace `your-username` with your actual GitHub username.
+:: manually running pre-commit on current changes
+:: note: by default pre-commit only runs on staged files
+> pre-commit run
 
-       set GITHUB_USERNAME=your-username
-       git clone [email protected]:%GITHUB_USERNAME%/conda "%CONDA_PROJECT_ROOT%"
-       cd "%CONDA_PROJECT_ROOT%"
-       git remote add upstream [email protected]:%GITHUB_USERNAME%/conda
+:: automatically running pre-commit during commit
+> git commit
+```
 
-3. Create a local development environment, and activate that environment
+Beware that some of the tools run by pre-commit can potentially modify the
+code (see [black](https://github.com/psf/black),
+[blacken-docs](https://github.com/asottile/blacken-docs), and
+[darker](https://github.com/akaihola/darker)). If pre-commit detects that any
+files were modified it will terminate the commit giving you the opportunity to
+review the code before committing again.
 
-       .\dev\start.bat
+Strictly speaking using pre-commit on your local machine for commits is
+optional (if you don't install pre-commit you will still be able to commit
+normally). But once you open a PR to contribue your changes, pre-commit will
+be automatically run at which point any errors that occur will need to be
+addressed prior to proceeding.
 
-   This command will create a project-specific base environment at `.\devenv`.
-   If the environment already exists, this command will just quickly activate
-   the already-created `.\devenv` environment.
+## Testing
 
-   To be sure that the conda code being interpreted is the code in the project
-   directory, look at the value of `conda location:` in the output of
-   `conda info --all`.
+We use pytest to run our test suite. Please consult pytest's
+[docs](https://docs.pytest.org/en/6.2.x/usage.html) for detailed instructions
+but generally speaking all you need is the following:
+
+**Bash (macOS, Linux, Windows)**
+
+```bash
+# reuse the development environment created above
+$ source ./dev/start
+
+# run conda's unit tests using GNU make
+$ make unit
+
+# or alternately with pytest
+$ pytest -m "not integration" conda tests
+
+# or you can use pytest to focus on one specific test
+$ pytest tests/test_create.py -k create_install_update_remove_smoketest
+```
+
+**cmd.exe (Windows)**
+
+```batch
+:: reuse the development environment created above
+> .\dev\start.bat
+
+:: run conda's unit tests with pytest
+> pytest -m "not integration" conda tests
+
+:: or you can use pytest to focus on one specific test
+> pytest tests\test_create.py -k create_install_update_remove_smoketest
+```
 
 ## Conda Contributor License Agreement
 

pyproject.toml

@@ -0,0 +1,9 @@
+[tool.black]
+line-length = 99
+target-version = ['py36', 'py37', 'py38']
+extend-exclude = '''
+^/(
+    conda/_vendor
+  | devenv
+)/
+'''

tests/requirements.txt

@@ -5,10 +5,11 @@ beautifulsoup4
 chardet
 conda
 conda-build
-conda-package-handling
-conda-verify
+conda-forge::pre-commit
 conda-forge::pytest-split
 conda-forge::xdoctest
+conda-package-handling
+conda-verify
 cytoolz
 filelock
 flake8
@@ -22,12 +23,12 @@ pkginfo
 pycosat
 pycrypto
 pyflakes
+pytest
 pytest-cov
 pytest-mock
 pytest-rerunfailures
 pytest-timeout
 pytest-xdist
-pytest
 requests
 responses
 ruamel_yaml