Applying mutation tests on shared

Playing around with mutation tests on shared. I think it's a good way to get our feet wet into the mutation testing world and understand what it means.

There is a lot of repetition between the mutations workflow and the testing one. But I don't want to damage the testing workflow because of mutation testing problems.

I am hoping we can later improve this with "workflows" from CircleCI: https://circleci.com/blog/persisting-data-in-workflows-when-to-use-caching-artifacts-and-workspaces/

But mutation tests is working. It only applies to lines present at the git diff (it takes way too long otherwise) and it produces interesting results which are currently saved on the "tests" tab on CircleCI: https://app.circleci.com/pipelines/github/codecov/shared/1239/workflows/82ebe8af-f5ac-4f09-ad01-8fcafd810aff/jobs/2518/tests

.circleci/config.yml

@@ -12,6 +12,29 @@ jobs:
             pip3 install black==19.10b0 isort
             black --check .
             isort --profile black --check .
+  mutations:
+    docker:
+      - image: circleci/python:3.7.9
+    working_directory: ~/repo
+    steps:
+      - checkout
+      - run:
+          name: Run Mutations
+          command: |
+            curl https://sh.rustup.rs -sSf | sh -s -- -y
+            source $HOME/.cargo/env
+            python3 -m venv venv
+            . venv/bin/activate
+            pip install setuptools_rust
+            python setup.py develop
+            git diff master $CIRCLE_SHA1 > data.patch
+            pip install mutmut[patch]
+            pip install -r tests/requirements.txt
+            mutmut run --use-patch-file data.patch || true
+            mkdir /tmp/artifacts;
+            mutmut junitxml > /tmp/artifacts/mut.xml
+      - store_test_results:
+          path: /tmp/artifacts/mut.xml
   build:
     docker:
       - image: circleci/python:3.7.9
@@ -79,6 +102,7 @@ workflows:
   version: 2
   lint-test:
     jobs:
+      - mutations
       - rust:
           filters:
             tags:

setup.cfg

@@ -0,0 +1,5 @@
+[mutmut]
+paths_to_mutate=shared/
+backup=False
+runner=python -m pytest -x
+tests_dir=tests/