Update: docs revamp

README.md

@@ -1,6 +1,15 @@
-# Causal Testing Framework: A Causal Inference-Driven Software Testing Framework
+# Causal Testing Framework
+### A Causal Inference-Driven Software Testing Framework
 
-![example workflow](https://github.com/CITCOM-project/CausalTestingFramework/actions/workflows/ci-tests.yaml/badge.svg) [![codecov](https://codecov.io/gh/CITCOM-project/CausalTestingFramework/branch/main/graph/badge.svg?token=04ijFVrb4a)](https://codecov.io/gh/CITCOM-project/CausalTestingFramework) [![Documentation Status](https://readthedocs.org/projects/causal-testing-framework/badge/?version=latest)](https://causal-testing-framework.readthedocs.io/en/latest/?badge=latest)
+
+[![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active) 
+![example workflow](https://github.com/CITCOM-project/CausalTestingFramework/actions/workflows/ci-tests.yaml/badge.svg) 
+[![codecov](https://codecov.io/gh/CITCOM-project/CausalTestingFramework/branch/main/graph/badge.svg?token=04ijFVrb4a)](https://codecov.io/gh/CITCOM-project/CausalTestingFramework) 
+[![Documentation Status](https://readthedocs.org/projects/causal-testing-framework/badge/?version=latest)](https://causal-testing-framework.readthedocs.io/en/latest/?badge=latest)
+![Dynamic TOML Badge](https://img.shields.io/badge/dynamic/toml?url=https%3A%2F%2Fraw.githubusercontent.com%2FCITCOM-project%2FCausalTestingFramework%2Fmain%2Fpyproject.toml&query=%24.project%5B'requires-python'%5D&label=python)
+![PyPI - Version](https://img.shields.io/pypi/v/causal-testing-framework)
+[![DOI](https://t.ly/FCT1B)](https://orda.shef.ac.uk/articles/software/CITCOM_Software_Release/24427516)
+![GitHub License](https://img.shields.io/github/license/CITCOM-project/CausalTestingFramework)
 
 Causal testing is a causal inference-driven framework for functional black-box testing. This framework utilises
 graphical causal inference (CI) techniques for the specification and functional testing of software from a black-box
@@ -12,10 +21,9 @@ system-under-test that is expected to cause a change to some output(s).
 
 ![Causal Testing Workflow](images/workflow.png)
 
-
 ## Installation
 
-See the readthedocs site for [installation
+See the Read the Docs site for [installation
 instructions](https://causal-testing-framework.readthedocs.io/en/latest/installation.html).
 
 ## Documentation

docs/README.md

@@ -6,7 +6,7 @@ The documentation is accessible on [Read the Docs](https://causal-testing-framew
 
 ## Locally building
 To build locally, the requirements in `docs/source/requirements.txt` will need to be installed.
-
+[README.md](README.md)
 Within `docs/`, run `make html` to create or update the .html files in the `docs/build` directory. 
 
 Running `make clean` will clean the `build` folder.  

docs/source/_static/css/custom.css

@@ -0,0 +1,29 @@
+/* Custom CSS */
+
+/* Note: /*
+The CSS below will only work if 'logo_only':True in conf.py, but will require adjusting if False.
+ */
+
+.wy-side-nav-search a .logo {
+    max-width: 200px;
+    max-height: 170px;
+    height: auto;
+    width:auto;
+}
+
+
+
+.wy-nav-top a {
+    color: #131E29;  /* Change the colour of the homepage navigation title on mobiles */
+}
+
+.wy-menu-vertical p.caption {
+
+    color: #9ADBE8; /* Change the colour of captions*/
+}
+
+
+.rst-content code.literal, .highlight .s2, .highlight .s1, .rst-content tt.literal{
+    color: #440099;
+    font-weight: bold;
+}

docs/source/conf.py

@@ -12,16 +12,17 @@
 #
 import os
 import sys
+from datetime import date
 
 # Include root directory of project
 sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../..")))
 
 
 # -- Project information -----------------------------------------------------
 
-project = "Causal Testing"
-copyright = "2021, Andrew Clark, Michael Foster, Neil Walkinshaw, Rob Hierons, Bob Turner"
-author = "Andrew Clark, Michael Foster, Neil Walkinshaw, Rob Hierons, Bob Turner"
+project = "Causal Testing Framework"
+copyright = f"{date.today().year}, The CITCOM Team"
+author = "Andrew Clark, Michael Foster, Neil Walkinshaw, Rob Hierons, Bob Turner, Christopher Wild, Farhad Allian"
 
 
 # -- General configuration ---------------------------------------------------
@@ -49,7 +50,24 @@
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-# html_static_path = ['_static']
+html_static_path = ['_static']
+
+html_css_files = ['css/custom.css']
 
 # Path to generate documentation from using sphinx AutoAPI
 autoapi_dirs = [os.path.abspath(os.path.join("..", "..", "causal_testing"))]
+
+autoapi_generate_api_docs = True
+autoapi_keep_files = True
+
+# Suppress label warnings
+suppress_warnings = ['autosectionlabel.*']
+
+
+html_logo = '_static/images/CITCOM-logo.png'
+
+html_theme_options = {
+    'style_nav_header_background': '#9ADBE8',  # Set the colour using CSS
+    'logo_only' : True,
+}
+

docs/source/credits.rst

@@ -0,0 +1,37 @@
+Acknowledging CITCOM
+*********************
+
+If you use the Causal Testing Framework for your work, please add the following line within your paper to acknowledge
+our work:
+
+.. admonition:: Important
+   :class: important
+
+   This research has used version X.Y.Z (software citation) of the Causal Testing Framework (paper citation).
+
+The paper citation should be the Causal Testing Framework paper_, and the software citation should be the specific
+Figshare DOI_ for the version used in your research.
+
+.. _paper: https://dl.acm.org/doi/10.1145/3607184
+.. _DOI: https://dl.acm.org/doi/10.1145/3607184
+
+.. code-block:: latex
+
+    @ARTICLE{Clark_etal_2023,
+    author = {Clark, Andrew G. and Foster, Michael and Prifling, Benedikt and Walkinshaw, Neil and Hierons, Robert M.
+    and Schmidt, Volker and Turner, Robert D.},
+    title = {Testing Causality in Scientific Modelling Software},
+    year = {2023},
+    publisher = {Association for Computing Machinery},
+    url = {https://doi.org/10.1145/3607184},
+    doi = {10.1145/3607184},
+    journal = {ACM Trans. Softw. Eng. Methodol.},
+    month = {jul},
+    keywords = {Software Testing, Causal Testing, Causal Inference}
+    }
+
+The CITCOM Team
+***************
+
+- Andrew G. Clark, Michael Foster, Richard Somers, Christopher Wild, Farhad Allian, Neil Walkinshaw.
+

docs/source/description.rst

@@ -1,29 +1,28 @@
-Framework Overview
+Background
 =====================================
 
-CTF Components
---------------
+The Causal Testing Framework consists of 3 main components: 1) Causal Specification, 2) Causal Test Case and 3) Data Collection.
 
 #.
-   :doc:`Causal Specification <../modules/causal_specification>`\ : Before we can test software, we need to obtain an
-   understanding of how it should behave in a particular use-case scenario. In addition, to apply graphical CI
+   :doc:`Causal Specification <../modules/causal_specification>`\ : To apply graphical CI
    techniques for testing, we need a causal DAG which depicts causal relationships amongst inputs and outputs. To
    collect this information, users must create a *causal specification*. This comprises a set of scenarios which place
    constraints over input variables that capture the use-case of interest, a causal DAG corresponding to this scenario,
    and a series of high-level functional requirements that the user wishes to test. In causal testing, these
    requirements should describe how the model should respond to interventions (changes made to the input configuration).
 
+
+
 #.
    :doc:`Causal Tests <../modules/causal_tests>`\ : With a causal specification in hand, we can now go about designing
    a series of test cases that interrogate the causal relationships of interest in the scenario-under-test. Informally,
-   a causal test case is a triple (M, X, Delta, Y), where M is the modelling scenario, X is an input configuration,
-   Delta is an intervention which should be applied to X, and Y is the expected *causal effect* of that intervention on
-   some output of interest. Therefore, a causal test case states the expected causal effect (Y) of a particular
-   intervention (Delta) made to an input configuration (X). For each scenario, the user should create a suite of causal
+   a causal test case is a triple ``(M, X, Delta, Y)``, where ``M`` is the modelling scenario, ``X`` is an input configuration,
+   ``Delta`` is an intervention which should be applied to ``X``, and ``Y`` is the expected *causal effect* of that intervention on
+   some output of interest. Therefore, a causal test case states the expected causal effect (``Y``) of a particular
+   intervention (``Delta``) made to an input configuration (``X``). For each scenario, the user should create a suite of causal
    tests. Once a causal test case has been defined, it is executed as follows:
 
-
-   #. Using the causal DAG, identify an estimand for the effect of the intervention on the output of interest. That is,
+   a. Using the causal DAG, identify an estimand for the effect of the intervention on the output of interest. That is,
       a statistical procedure capable of estimating the causal effect of the intervention on the output.
    #. Collect the data to which the statistical procedure will be applied (see Data collection below).
    #. Apply a statistical model (e.g. linear regression or causal forest) to the data to obtain a point estimate for
@@ -35,6 +34,8 @@ CTF Components
       test should pass or fail based on the results. In the simplest case, this takes the form of an assertion which
       compares the point estimate to the expected causal effect specified in the causal test case.
 
+
+
 #.
    :doc:`Data Collection <../modules/data_collector>`\ : Data for the system-under-test can be collected in two
    ways: experimentally or observationally. The former involves executing the system-under-test under controlled
@@ -48,22 +49,3 @@ CTF Components
    provide instructions for an execution that will fill the gap (future work).
 
 For more information on each of these steps, follow the link to their respective documentation.
-
-Causal Inference Terminology
-----------------------------
-
-Here are some explanations for the causal inference terminology used above.
-
-
-* Causal inference (CI) is a family of statistical techniques designed to quantify and establish **causal**
-  relationships in data. In contrast to purely statistical techniques that are driven by associations in data, CI
-  incorporates knowledge about the data-generating mechanisms behind relationships in data to derive causal conclusions.
-* One of the key advantages of CI is that it is possible to answer causal questions using **observational data**. That
-  is, data which has been passively observed rather than collected from an experiment and, therefore, may contain all
-  kinds of bias. In a testing context, we would like to leverage this advantage to test causal relationships in software
-  without having to run costly experiments.
-* There are many forms of CI techniques with slightly different aims, but in this framework we focus on graphical CI
-  techniques that use directed acyclic graphs to obtain causal estimates. These approaches used a causal DAG to explain
-  the causal relationships that exist in data and, based on the structure of this graph, design statistical experiments
-  capable of estimating the causal effect of a particular intervention or action, such as taking a drug or changing the
-  value of an input variable.

docs/source/dev/actions_and_webhooks.rst

@@ -1,26 +1,30 @@
 Github Actions and Webhooks
 ===========================
 
-Github Actions
+Actions
 --------------
 
-Currently this project makes use of 3 `Github Actions <https://github.com/features/actions>`_, which can be found in the `.github/workflows <https://github.com/CITCOM-project/CausalTestingFramework/tree/main/.github/workflows>`_ directory
+Currently, this project makes use of 3 `Github Actions <https://github.com/features/actions>`_,
+which can be found in the
+`.github/workflows <https://github.com/CITCOM-project/CausalTestingFramework/tree/main/.github/workflows>`_ directory.
 
 They are:
 
-#.  ci-tests.yaml, which runs CI tests on each PR
+#.  ``ci-tests.yaml``, which runs continuous integration (CI) tests on each on each pull request.
 
-#.  lint-format.yaml, runs linting on each PR
+#.  ``lint-format.yaml``, which runs linting on each pull request.
 
-#.  publish-to-pypi.yaml, runs when a new version tag is pushed and publishes that tag version to pypi
+#.  ``publish-to-pypi.yaml``, runs when a new version tag is pushed and publishes that tag version to PyPI.
 
-Github Webhooks
+Webhooks
 ---------------
 
-The project also uses 3 `Webhooks <https://docs.github.com/en/webhooks-and-events/webhooks/about-webhooks>`_, which can be found in the `project settings <https://github.com/CITCOM-project/CausalTestingFramework/settings>`_ on Github
+The project also uses 3 `Webhooks <https://docs.github.com/en/webhooks-and-events/webhooks/about-webhooks>`_, which can
+be found in the `project settings <https://github.com/CITCOM-project/CausalTestingFramework/settings>`_ on Github. These
+include:
 
-#.  To codacy
+#.  `Codacy <https://github.com/codacy>`_
 
-#.  To codecov
+#.  `Codecov <https://github.com/codecov>`_
 
-#.  To readthedocs
+#.  `Read the Docs <https://github.com/readthedocs>`_

docs/source/dev/documentation.rst

@@ -1,27 +1,23 @@
 Project Documentation
 =====================
 
-This page aims to describe:
+This page describes:
 
-#. The projects documentation style
+#. The projects documentation style,
 
-#. The tools used for documentation
+#. The tools used for documentation,
 
-#. ReadTheDocs where the documentation for this project is hosted
+#. Read the Docs where the documentation for this project is hosted.
 
 
-Documentation Style
+Style and Tools
 -------------------
 
-The `Sphinx docstring format <https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html#the-sphinx-docstring-format>`_ is used throughout the projects codebase to allow for the easy understanding of classes, methods, functions, etc.
-This format allows for the easy generation of html documentation pages.
+The `Sphinx docstring format <https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html#the-sphinx-docstring-format>`_ is used
+throughout the projects codebase to allow for the easy understanding of classes, methods, functions, etc. This format
+allows for the easy generation of html documentation pages. It also checks for docstrings that have been added to the projects PyLint configuration.
 
-Checks for docstrings have been added to the projects Pylint configuration.
-
-Documentation Tools
--------------------
-
-To install the packages required to work with the documentation please ensure the projects **dev** dependencies are installed::
+To install the packages required to work with the documentation, please ensure the projects **dev** dependencies are installed::
 
     pip install causal-testing-framework[dev]
 
@@ -42,20 +38,25 @@ To cleanup the compiled docs you can run::
     make clean
 
 
-Situation within `docs/source` are the reStructuredText files (.rst) which contain the handwritten doc pages, which get compiled by the make commands.
 
-Autodoc & Autoapi
+The source reStructuredText files (.rst) files can be found within `docs/source` directory, which contains the
+handwritten documentation pages and get compiled by the ``make`` commands.
+
+Autodoc & AutoAPI
 *****************
 
-`Autodoc <https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html>`_ is an extension to sphinx that can import code modules and compile documentation from their docstrings.
+`Autodoc <https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html>`_ is an extension to Sphinx that can import
+code modules and compile documentation from their docstrings.
 
-`AutoAPI <https://sphinx-autoapi.readthedocs.io/en/latest/>`_ is a third party sphinx tool for recursively discovering code modules and compiling them into a logical doctree structure
+`AutoAPI <https://sphinx-autoapi.readthedocs.io/en/latest/>`_ is a third party Sphinx tool for recursively discovering
+code modules and compiling them into a logical doctree structure
 
-The configuration for Sphinx, Autodoc and AutoAPI are all found in `/docs/source/conf.py <https://github.com/CITCOM-project/CausalTestingFramework/blob/main/docs/source/conf.py>`_.
+The configuration for Sphinx, Autodoc and AutoAPI can all be found in `/docs/source/conf.py <https://github.com/CITCOM-project/CausalTestingFramework/blob/main/docs/source/conf.py>`_.
 
 ReadTheDocs
 -----------
-`Read the Docs <https://readthedocs.org/>`_ is a documentation hosting site that hosts, versions and builds documentation for free for open source projects.
+`Read the Docs <https://readthedocs.org/>`_ is a documentation hosting site that hosts, versions and builds documentation
+for free for open source projects.
 
-This project makes use of a Github Webook to trigger the build in ReadTheDocs, further reading on this can be found :doc:`here <../dev/actions_and_webhooks>`\
+This project makes use of a Github Webhook to trigger the build in ReadTheDocs, further reading on this can be found :doc:`here <../dev/actions_and_webhooks>`\
 

docs/source/dev/version_release.rst

@@ -2,28 +2,28 @@ Version Releases
 ================
 
 This project follows the `Semantic Versioning 2.0.0 <https://semver.org/>`_ style for code releases.
-This page aims to describe the release process for a new code version on the `Project Github <https://github.com/CITCOM-project/CausalTestingFramework>`_
+This page describes the release process for a new framework version on the `Project Github <https://github.com/CITCOM-project/CausalTestingFramework>`_.
 
 How to release
 --------------
 
-#. Once your PR(s) are merged, navigate to https://github.com/CITCOM-project/CausalTestingFramework/releases, which can be found on the right hand side of the projects Github main page by clicking on 'Releases'
+#. Once your PR(s) are merged, navigate to https://github.com/CITCOM-project/CausalTestingFramework/releases, which can be found on the right hand side of the projects Github main page by clicking on 'Releases'.
 
-#. Press the **Draft a new release** button in the top right of the releases page
+#. Press the **Draft a new release** button in the top right of the releases page.
 
-#. Press the **Choose a tag** button and add the new version following the Semantic Version guidelines. Please include the 'v' before the tag, e.g. **v0.0.0**
+#. Press the **Choose a tag** button and add the new version following the Semantic Version guidelines. Please include the 'v' before the tag, e.g. **v0.0.0**.
 
-#. Enter the same tag name into the **Release Title** box
+#. Enter the same tag name into the **Release Title** box.
 
-#. Press **Generate Release Notes** button
+#. Press **Generate Release Notes** button.
 
 #. Add any additional information that may be helpful to the release notes. If there are breaking changes for example, which modules will they affect?
 
-#. Ensure the **Set as the latest release** checkbox is selected
+#. Ensure the **Set as the latest release** checkbox is selected.
 
-#. Press publish release
+#. Press publish release.
 
-#. Check that the Github Action worker, found in the `Actions tab <https://github.com/CITCOM-project/CausalTestingFramework/actions>`_ on the github, has successfully completed. Typical time to publish to PyPI is around 2 minutes.
+#. Check that the Github Action worker, found in the `Actions tab <https://github.com/CITCOM-project/CausalTestingFramework/actions>`_ has successfully completed. The typical time to publish to PyPI is around 2 minutes.
 
 #. Check on the projects `PyPI page <https://pypi.org/project/causal-testing-framework/>`_ that the latest release is ready!