Automatic documentation (cta-observatory#130)

* Follow template from repository https://github.com/cta-observatory/project-template-python-pure

* Try to implement first auto-generated doc with Sphinx

* Accept PNG for logo in doc

* Try to deploy GitHub pages in CI

* Try to include all imported sub-modules in API doc.

* Try doc with sphinx-automodapi

* Add sphinx-automodapi as dependency

* Try doc with sphinx-automodapi

* Try doc with autosummary

* Try doc piece by piece

* Try doc again

* Try doc with autosummary

* Try doc with sphinx-autoapi

* Revert "Try doc with autosummary"

This reverts commit 9d09952.

* Revert "Try doc with sphinx-autoapi"

This reverts commit 5baf409.

* Simplify doc conf

* Back to pydata sphinx theme

* The alabaster theme works for the doc!

* Revert "The alabaster theme works for the doc!"

This reverts commit c295b6b.

* Revert "Back to pydata sphinx theme"

This reverts commit 4ad177d.

* Revert "Simplify doc conf"

This reverts commit af71ab4.

* Revert "Revert "Try doc with sphinx-autoapi""

This reverts commit 7f9a0b4.

* Revert "Revert "Try doc with autosummary""

This reverts commit 14c400f.

* Revert "Try doc with sphinx-autoapi"

This reverts commit 5baf409.

* Revert "Try doc with autosummary"

This reverts commit 9d09952.

* Revert "Try doc with autosummary"

This reverts commit f3084a6.

* Revert "Try doc again"

This reverts commit 47d5b33.

* Revert "Disable autosummary in doc"

This reverts commit 6a2203e.

* Revert "Try doc piece by piece"

This reverts commit 372b1f3.

* Revert "Try doc with autosummary"

This reverts commit 9fbb84b.

* Revert "Try doc with sphinx-automodapi"

This reverts commit 25a8a45.

* Revert "Try doc with sphinx-automodapi"

This reverts commit 885e41f.

* Revert "Add sphinx-automodapi as dependency"

This reverts commit 69431a2.

* Revert "Try doc with sphinx-automodapi"

This reverts commit c15670d.

* Re-structure API documentation, using automodapi

* Fix docstrings

* Blank line missing in doc for Sphinx & pre-commit hooks

* Alabaster theme

* Add more doc

* Work on Sphinx configuration.

* Add configuration for Read the Docs.

* Avoid building all branches on push

* Only deploy github-pages (if we want to) on push in main branch

* Remove Read the Docs configuration, to deploy doc as GitHub pages only.

* Add more auto-docs for some submodules.

* Lint code

* Restore chargesComponent.py from main

* Try to solve errors for docs in CI

* Try to solve errors for docs in CI

* Fix typo

* Try to fix errors in docs in CI

* Try to fix errors in docs in CI

* Test excluding components from doc

* Try fixing doc for makers submodule

* Try fixing doc for makers submodule

* Try fixing doc for makers submodule

* Try fixing doc for makers submodule

* Fix clean in Makefile

* Try to deploy with alabaster theme.

* Force a doc CI run in ultra-verbose mode, to understand why the docs are not properly built in CI.

* Makefile for Sphinx back to initial state

* Force a doc CI run in ultra-verbose mode

* Force a doc CI run in verbose mode

* Do not explicitly pass options to sphinx in CI, but rely on docs Makefile itself.

* Only use automodule with Sphinx

* Fix typo in docstring.

* Use Sphinx nitpick_ignore

* Force a test deployment of github-pages

* Fix typo and force a test deployment of github-pages

* Deploy doc using token

* Add write permission to docs CI

* Changed syntax for arguments to github-pages-deploy GitHub action.

* Changed syntax for arguments to github-pages-deploy GitHub action.

* Play with github-pages-deploy GitHub action

* Force doc deployment

* Only deploy doc on PR on main branch

* Force test deployment of doc

* Force test deployment of doc

* Force test deployment of doc

* Force test deployment of doc

* Force test deployment of doc

* Force test deployment of doc

* Force test deployment of doc

* Force version of github-pages-deploy action to last one.

* Test forcing doc deployment

* Deploy doc on Read the Docs instead of GitHub pages

* Change doc target recipe to docs, needed, apparently, to deploy on Read the Docs.

* Add a skeleton for a developer guide.

---------

Co-authored-by: Jean-Philippe Lenain <[email protected]>

.github/release-drafter.yml

@@ -1,4 +1,8 @@
 template: |
-  ## What’s Changed
+  ## What’s Changed since $PREVIOUS_TAG
   
   $CHANGES
+  
+  ## Contributors
+  
+  $CONTRIBUTORS

.github/workflows/ci.yml

@@ -3,10 +3,12 @@ name: CI
 # by not building all branches on push, we avoid the duplicated builds in PRs
 on:
   push:
-    branches:
-      - main
-    tags:
-      - '**'
+  # TODO
+  # Remove comment: just a test to force CI on push
+  # branches:
+  #   - main
+  # tags:
+  #   - '**'
   pull_request:
 
 env:
@@ -111,3 +113,27 @@ jobs:
           pytest -n auto --dist loadscope --cov=nectarchain --cov-report=xml
 
       - uses: codecov/codecov-action@v4
+
+  docs:
+    needs: lint
+    permissions:
+      contents: write
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.9
+
+      - name: Install doc dependencies
+        run: |
+          pip install -e .[docs]
+          # git describe --tags
+          python -c 'import nectarchain; print(nectarchain.__version__)'
+
+      - name: Build docs
+        run: make -C docs html

.github/workflows/release-drafter.yml

@@ -2,13 +2,15 @@ name: Release Drafter
 
 on:
   push:
+    # branches to consider in the event; optional, defaults to all
     branches:
       - main
 
 jobs:
   update_release_draft:
     runs-on: ubuntu-latest
     steps:
+      # Drafts your next Release notes as Pull Requests are merged into "master"
       - uses: release-drafter/release-drafter@v6
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.gitignore

@@ -92,7 +92,6 @@ src/nectarchain/user_scripts/**/test
 **.log
 **.pdf
 **.csv
-**.png
 
 #VScode
 .vscode/

.readthedocs.yaml

@@ -0,0 +1,35 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the OS, Python version and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"
+    # You can also specify other tool versions:
+    # nodejs: "19"
+    # rust: "1.64"
+    # golang: "1.19"
+
+# Build documentation in the "docs/" directory with Sphinx
+sphinx:
+  configuration: docs/conf.py
+
+# Optionally build your docs in additional formats such as PDF and ePub
+# formats:
+#    - pdf
+#    - epub
+
+# Optional but recommended, declare the Python requirements required
+# to build your documentation
+# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
+python:
+  install:
+    - method: pip
+      path: .
+      extra_requirements:
+        - docs

docs/Makefile

@@ -0,0 +1,24 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?= -W --keep-going -n --color -j auto
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+clean:
+	rm -rf api
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/api-reference/data/index.rst

@@ -0,0 +1,11 @@
+.. _data:
+
+Data (`data`)
+=============
+Module containing functions and classes holding NectarCAM data structure, and interaction with DIRAC.
+
+Reference/API
+=============
+.. automodule:: nectarchain.data
+   :members:
+   :undoc-members:

docs/api-reference/display/index.rst

@@ -0,0 +1,10 @@
+.. _display:
+
+Display (`display`)
+===================
+
+Reference/API
+=============
+.. automodule:: nectarchain.display
+   :members:
+   :undoc-members:

docs/api-reference/dqm/index.rst

@@ -0,0 +1,12 @@
+.. _dqm:
+
+Data Quality Monitoring (`dqm`)
+===============================
+
+Module for the NectarCAM data quality monitoring.
+
+Reference/API
+=============
+.. automodule:: nectarchain.dqm
+   :members:
+   :undoc-members:

docs/api-reference/index.rst

@@ -0,0 +1,8 @@
+API Docs
+========
+
+.. toctree::
+  :maxdepth: 1
+  :glob:
+
+  */index

docs/api-reference/tools/index.rst

@@ -0,0 +1,12 @@
+.. _tools:
+
+Tools (`tools`)
+===============
+
+Tools submodule.
+
+Reference/API
+=============
+.. automodule:: nectarchain.tools
+   :members:
+   :undoc-members:

docs/api-reference/utils/index.rst

@@ -0,0 +1,12 @@
+.. _utils:
+
+Utilities (`utils`)
+===================
+
+Utilities submodule.
+
+Reference/API
+=============
+.. automodule:: nectarchain.utils
+   :members:
+   :undoc-members:

docs/conf.py

@@ -0,0 +1,138 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+import datetime
+import sys
+from pathlib import Path
+
+import nectarchain
+
+if sys.version_info < (3, 11):
+    import tomli as tomllib
+else:
+    import tomllib
+
+pyproject_path = Path(__file__).parent.parent / "pyproject.toml"
+pyproject = tomllib.loads(pyproject_path.read_text())
+
+project = pyproject["project"]["name"]
+author = pyproject["project"]["authors"][0]["name"]
+copyright = "{}.  Last updated {}".format(
+    author, datetime.datetime.now().strftime("%d %b %Y %H:%M")
+)
+python_requires = pyproject["project"]["requires-python"]
+
+# make some variables available to each page
+rst_epilog = f"""
+.. |python_requires| replace:: {python_requires}
+"""
+
+version = nectarchain.__version__
+# The full version, including alpha/beta/rc tags.
+release = version
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    "sphinx.ext.githubpages",
+    "sphinx.ext.intersphinx",
+    "sphinx.ext.autodoc",
+    "sphinx.ext.autosummary",
+    "sphinx.ext.viewcode",
+    "sphinx_automodapi.automodapi",
+    "sphinx_automodapi.smart_resolver",
+    "numpydoc",
+]
+
+numpydoc_show_class_members = False
+autosummary_generate = True
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+# source_suffix = ['.rst', '.md']
+source_suffix = ".rst"
+
+# The master toctree document.
+master_doc = "index"
+
+templates_path = []  # ["_templates"]
+
+exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
+
+# intersphinx allows referencing other packages sphinx docs
+intersphinx_mapping = {
+    "python": ("https://docs.python.org/3.9", None),
+    "astropy": ("https://docs.astropy.org/en/latest/", None),
+    "ctapipe": ("https://ctapipe.readthedocs.io/en/v0.19.3/", None),
+    "matplotlib": ("https://matplotlib.org/", None),
+    "traitlets": ("https://traitlets.readthedocs.io/en/stable/", None),
+}
+
+# These links are ignored in the checks, necessary due to broken intersphinx for these
+nitpick_ignore = [
+    ("py:obj", "enum.auto"),
+    ("py:obj", "enum.IntFlag"),
+    ("py:obj", "enum.unique"),
+    # coming from inherited traitlets docs
+    ("py:class", "t.Union"),
+    ("py:class", "t.Dict"),
+    ("py:class", "t.Tuple"),
+    ("py:class", "t.List"),
+    ("py:class", "t.Any"),
+    ("py:class", "t.Type"),
+    ("py:class", "Config"),
+    ("py:class", "Unicode"),
+    ("py:class", "StrDict"),
+    ("py:class", "ClassesType"),
+]
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = True
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = "pydata_sphinx_theme"
+# html_theme = "alabaster"
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = f"{project}doc"
+
+html_logo = "_static/nectarcam.png"
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ["_static"]
+html_context = {
+    "default_mode": "light",
+    "github_user": "cta-observatory",
+    "github_repo": f"{project}",
+    "github_version": "main",
+    "doc_path": "docs",
+}
+html_file_suffix = ".html"
+
+html_theme_options = {
+    "navigation_with_keys": False,
+    "github_url": f"https://github.com/cta-observatory/{project}",
+    "navbar_start": ["navbar-logo", "version-switcher"],
+    "announcement": """
+        <p>nectarchain is not stable yet, so expect large and rapid
+        changes to structure and functionality as we explore various
+        design choices before the 1.0 release.</p>
+    """,
+}
+
+# The name for this set of Sphinx documents.  If None, it defaults to
+# "<project> v<release> documentation".
+# html_title = f"{project} v{release}"

docs/developer-guide/getting-started.rst

@@ -0,0 +1,19 @@
+.. _getting_started_dev:
+
+Getting Started For Developers
+==============================
+
+Building the documentation
+--------------------------
+
+To locally build the documentation, optional dependencies should be installed with:
+
+.. code-block::
+
+    $ pip install -e ".[docs]"
+
+The documentation can then be compiled with:
+
+.. code-block::
+
+    $ make -C docs html

docs/developer-guide/index.rst

@@ -0,0 +1,9 @@
+.. _developer-guide:
+
+Developer Guide
+===============
+
+.. toctree::
+   :maxdepth: 2
+
+   getting-started