Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github actions and ReadTheDocs integration #262

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
689ee4d
Add workflow to render documentation
aperezhortal Feb 5, 2022
523153f
Fix requirements.txt path
aperezhortal Feb 5, 2022
2a15d46
Add PYSTEPS_DATA_PATH
aperezhortal Feb 5, 2022
b125626
Fix working dir for pysteps installation
aperezhortal Feb 5, 2022
2976acc
Add option to silent the dataset download progress
aperezhortal Feb 5, 2022
b7163d6
Update working dir for download data step
aperezhortal Feb 5, 2022
efcb306
Replace ./docs -> ./doc
aperezhortal Feb 5, 2022
3cf90ad
Install pysteps as development to build extensions in place.
aperezhortal Feb 5, 2022
f325d90
Set PYSTEPSRC
aperezhortal Feb 5, 2022
66b8eee
Tar artifact files to preserve files names and properties
aperezhortal Feb 5, 2022
9a7c0df
Upload auto_examples.tar artifact
aperezhortal Feb 5, 2022
150fe51
Do not tar intermediate dirs
aperezhortal Feb 5, 2022
353000d
Fix title underline
aperezhortal Feb 5, 2022
38c770b
Use artifact with rendered gallery in RTD
aperezhortal Feb 5, 2022
a281a45
Only download the artifacts in RTD, otherwise render gallery
aperezhortal Feb 5, 2022
2f2cde5
Add numpydoc to docs requirements
aperezhortal Feb 5, 2022
4523633
Add requests library to doc requirements
aperezhortal Feb 5, 2022
5cb9066
Fix auto-examples dir
aperezhortal Feb 5, 2022
203c01a
Fix docstrings
aperezhortal Feb 5, 2022
793341d
Add RTD manual trigger
aperezhortal Feb 6, 2022
7f29bc7
Update repo URL temporary to work on the fork
aperezhortal Feb 6, 2022
799ea46
Fix RTD destination branch
aperezhortal Feb 6, 2022
1e3b617
Update pre-commit hook config
aperezhortal Feb 6, 2022
359680f
Stylistic adjustments
aperezhortal Feb 6, 2022
933ecec
Abort documentation build examples error
aperezhortal Feb 26, 2022
adec505
Save conda environment in artifact
aperezhortal Feb 26, 2022
dc46166
Fix export path
aperezhortal Feb 26, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 147 additions & 0 deletions .github/workflows/render_example_gallery.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# Action to render the example gallery, upload them as artifacts. After the examples are
# rendered, the ReadTheDocs build is triggered pulling the artifacts with the rendered
# example gallery.
#
# This workflow is inspired by the actions in the https://github.com/dfm/rtds-action and
# https://github.com/wradlib/wradlib-notebooks projects.
#
# Steps needed to setup this workflow
# ===================================
#
# Read-the-docs configuration
# ---------------------------
#
# 1. In Admin/Integrations, add a generic API incoming webhook.
# Save the webhook url and token value.
# 2. Add the RTD_GITHUB_TOKEN environmental variable in RTD (Settings/Environment Variables)
# with the pysteps-bot's Github access token. This access token should only
# have the "public_repo" scope.
# At the moment, this token is needed to download the artifacts in RTD's.
#
# Github configuration
# ---------------------
#
# 1. Add the following secrets to the pysteps repository:
# - RTD_WEBHOOK_URL: Read-the-docs webhook. Important: the RTD-Github integration
# must not be activated since this workflow triggers the RTD build.
# - RTD_TOKEN: RTD webhook access token.

name: Render the example gallery

on:
# Triggers the workflow on push or pull request events to the master branch
push:
branches: [ master, rtd_using_gh_artifacts ]
# pull_request:
# branches: [ master ]
release:
types: [ published ]

jobs:
render_example_gallery:
name: Render the example gallery
runs-on: "ubuntu-latest"

defaults:
run:
shell: bash -l {0}

outputs:
rtd_branch: ${{steps.set_rendered_branch.outputs.render_branch}}
gallery_branch: ${{steps.set_rendered_branch.outputs.render_branch}}

steps:
- name: Checkout repository
uses: actions/checkout@v2

- name: Install mamba and create environment
uses: mamba-org/provision-with-micromamba@main
with:
environment-file: ci/ci_test_env.yml
extra-specs: python=3.8
environment-name: doc_builder

- name: Install pygrib
run: mamba install --quiet pygrib

- name: Install sphinx dependencies using pip
run: |
pip install -r doc/requirements.txt

- name: Install pysteps
working-directory: ${{github.workspace}}
run: |
pip install -e .

- name: Install pysteps-data
env:
PYSTEPS_DATA_PATH: ${{github.workspace}}/pysteps_data
working-directory: ${{github.workspace}}/ci
run: |
python fetch_pysteps_data.py
python -c "import pysteps; print(pysteps.config_fname())"

- name: Build example gallery
working-directory: ./doc
env:
PYSTEPSRC: ${{github.workspace}}/pysteps_data/pystepsrc
run: |
make html

- name: Export conda environment
run: conda env export > doc/source/auto_examples/environment.yml

- name: Tar auto_examples files
# Needed to maintain permissions and case-sensitive files
# https://github.com/actions/upload-artifact#maintaining-file-permissions-and-case-sensitive-files
run: tar -cvf auto_examples.tar -C doc/source/auto_examples .

- uses: actions/upload-artifact@v2
with:
name: auto_examples-for-${{ github.sha }}
path: auto_examples.tar

trigger_rtd:
# Task inspired in the following wradlib's action:
# https://github.com/wradlib/wradlib-notebooks/blob/main/.github/workflows/render_notebooks.yml
needs: [ render_example_gallery ]
name: Trigger readthedocs
runs-on: ubuntu-latest
defaults:
run:
shell: bash -l {0}
steps:
- name: Determine rendered branch name
id: set_rtd_branch
run: |
if [[ "${GITHUB_EVENT_NAME}" == "push" ]]; then
rtd_branch="${GITHUB_REF##*/}"
elif [[ "${GITHUB_EVENT_NAME}" == "pull_request" ]]; then
rtd_branch=$GITHUB_BASE_REF
elif [[ "${GITHUB_REF_TYPE}" == "tag" ]]; then
rtd_branch="${GITHUB_REF_NAME}"
else
rtd_branch="undefined"
fi

echo "::set-output name=rtd_branch::${rtd_branch}"

- name: Print debug information
env:
rtd_branch: ${{steps.set_rtd_branch.outputs.rtd_branch}}
run: |
echo "GITHUB_REF=${GITHUB_REF}"
echo "GITHUB_REF_TYPE=${GITHUB_REF_TYPE}"
echo "GITHUB_REF_NAME=${GITHUB_REF_NAME}"
echo "GITHUB_EVENT_NAME=${GITHUB_EVENT_NAME}"
echo "rtd_branch=${rtd_branch}"

- name: Trigger readthedocs for the corresponding branch
env:
RTD_TOKEN: ${{ secrets.RTD_TOKEN }}
RTD_WEBHOOK_URL: ${{ secrets.RTD_WEBHOOK_URL }}
RTD_BRANCH: ${{steps.set_rtd_branch.outputs.rtd_branch}}
run: |
if [[ ! -z "${RTD_BRANCH}" ]]; then
curl -X POST -d "branches=${RTD_BRANCH}" -d "token=${RTD_TOKEN}" "${RTD_WEBHOOK_URL}"
fi
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/psf/black
rev: 21.6b0
rev: 22.1.0
hooks:
- id: black
language_version: python3
2 changes: 1 addition & 1 deletion ci/fetch_pysteps_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

tox_test_data_dir = os.environ["PYSTEPS_DATA_PATH"]

download_pysteps_data(tox_test_data_dir, force=True)
download_pysteps_data(tox_test_data_dir, force=True, show_progress=False)

create_default_pystepsrc(
tox_test_data_dir, config_dir=tox_test_data_dir, file_name="pystepsrc"
Expand Down
4 changes: 3 additions & 1 deletion doc/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Additional requeriments related to the documentation build only
# Additional requirements related to the documentation build only
sphinx
requests
numpydoc
sphinxcontrib.bibtex
sphinx-book-theme
sphinx_gallery
Expand Down
Loading