Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add sphinx project's basic configuration and structure to deploy on RTD #17

Merged
merged 16 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Docs bundle

build
19 changes: 19 additions & 0 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# .readthedocs.yml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Set the version of Python and other tools you might need
version: 2
build:
os: ubuntu-22.04
tools:
python: "3.11"

# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: docs/source/conf.py
fail_on_warning: true

python:
install:
- requirements: requirements/doc.txt
30 changes: 30 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
.PHONY: clean help \
requirements selfcheck upgrade docs

.DEFAULT_GOAL := help

# For opening files in a browser. Use like: $(BROWSER)relative/path/to/file.html
BROWSER := python -m webbrowser file://$(CURDIR)/

help: ## display this help message
@echo "Please use \`make <target>' where <target> is one of"
@awk -F ':.*?## ' '/^[a-zA-Z]/ && NF==2 {printf "\033[36m %-25s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST) | sort

# Define PIP_COMPILE_OPTS=-v to get more information during make upgrade.
PIP_COMPILE = pip-compile --upgrade $(PIP_COMPILE_OPTS)

upgrade: export CUSTOM_COMPILE_COMMAND=make upgrade
upgrade: ## update the requirements/*.txt files with the latest packages satisfying requirements/*.in
pip install -r requirements/pip-tools.txt
# Make sure to compile files after any other files they include!
$(PIP_COMPILE) -o requirements/pip-tools.txt requirements/pip-tools.in
$(PIP_COMPILE) -o requirements/doc.txt requirements/doc.in

requirements: ## install development environment requirements
pip install -r requirements/pip-tools.txt -r requirements/doc.txt

docs: ## build the documentation
cd docs && $(MAKE) html

MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
selfcheck: ## check that the Makefile is well-formed
@echo "The Makefile is well-formed."
23 changes: 23 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = source
BUILDDIR = build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

serve_docs: ## serve the built docs locally to preview the site in the browser
python -m http.server 8200 --directory build/html
35 changes: 35 additions & 0 deletions docs/make.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
@ECHO OFF

pushd %~dp0

REM Command file for Sphinx documentation

if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=source
set BUILDDIR=build

if "%1" == "" goto help

%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.https://www.sphinx-doc.org/
exit /b 1
)

%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
goto end

:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%

:end
popd
Binary file added docs/source/_static/Knowledge_Base_logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
93 changes: 93 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

# -- Path setup --------------------------------------------------------------

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))


# -- Project information -----------------------------------------------------

project = 'Picasso Workflow'
copyright = '2024, edunext'
author = 'edunext'


# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

extensions = [
"sphinxcontrib.youtube",
"sphinxcontrib.images",
"sphinx_panels",
"sphinxcontrib.contentui",
"sphinx_copybutton",
"sphinx.ext.graphviz",
"sphinxcontrib.mermaid",
"recommonmark",
]

templates_path = ['_templates']
exclude_patterns = []

mariajgrimaldi marked this conversation as resolved.
Show resolved Hide resolved


# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output

html_theme = 'sphinx_book_theme'
html_static_path = ['_static']

# eduNEXT configuration
html_logo = '_static/Knowledge_Base_logo.png'
extra_navbar_content = """
<hr/><br/>

<a class='external' href='https://github.com/eduNEXT/picasso' target='_blank'>About the
project in Github</a><br/>
"""

html_theme_options = {
"repository_url": "https://github.com/eduNEXT/picasso",
"repository_branch": "main",
"path_to_docs": "source",
"use_edit_page_button": True,
"logo_only": False,
"extra_navbar": extra_navbar_content,
"home_page_in_toc": True,
"show_navbar_depth": 1,
"use_repository_button": True,
"use_issues_button": True
}

html_sidebars = {'**': ["sidebar-logo.html", "search-field.html", "sbt-sidebar-nav.html"]}

# For custom styles
images_config = {
"default_image_width": "100%",
}

html_css_files = [
"css/custom.css",
"https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css",
]

html_js_files = [
"_js/custom.js",
]

togglebutton_hint = "Show"
togglebutton_hint_hide = "Hide"

# Panels conf
panels_add_bootstrap_css = False
# html_title = ""
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Configuring Docker Registries
###################################
Configure a Different Docker Registry
#####################################

In Tutor, you can configure docker registries by setting the variable ``DOCKER_REGISTRY`` in the ``config.yml`` file. This variable is used to specify the registry where the images will be pushed. By default, the images are pushed to the Docker Hub registry ``docker.io/``.

Expand All @@ -9,7 +9,10 @@ To allow further customization in the build process, Picasso Workflow also suppo
2. Setting the corresponding secrets in the Github Actions repository's secrets settings. If you're using Docker Hub, you should set the ``DOCKERHUB_USERNAME`` and ``DOCKERHUB_PASSWORD`` secrets. If you're using AWS ECR, you should set the ``AWS_ACCESS_KEY_ID``, ``AWS_SECRET_ACCESS_KEY``, and ``AWS_REGION`` secrets.
3. Using the Picasso Workflow with the inputs and secrets set.

Here is an example of how to configure the Docker registries in the ``config.yml`` file:
AWS ECR registry
================

Here is an example of how to configure the Docker registries in the ``config.yml`` file to use the AWS ECR registry:

.. code-block:: yaml

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Picasso Workflow Configurations
###################################
Configure Picasso Workflow
###########################

Picasso is a `Github Workflow`_ implemented to be flexible enough to be adopted by different organizations with their own requirements. This document will guide you through the different configurations available to you.
Picasso is a `Github Workflow`_ implemented to be flexible enough to be adopted by different organizations with their own requirements. This document will guide you through the different configurations available to you when using the Picasso workflow in your repository.

.. _`Github Workflow`: https://docs.github.com/en/actions/sharing-automations/reusing-workflows

Expand All @@ -18,7 +18,7 @@ Inputs are the parameters that you can set to customize the behavior of the Pica

These inputs can be set in the workflow file that calls the Picasso workflow using the ``with`` keyword, by manually setting them in the workflow file, or by using the ``workflow_dispatch`` event. For more details on how to set input values in Github Actions, please refer to the `Workflow syntax for GitHub Actions`_ documentation.

.. `Workflow syntax for GitHub Actions`: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions
.. _`Workflow syntax for GitHub Actions`: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions

Here is an example of how to use the Picasso workflow with the inputs set:

Expand Down
11 changes: 11 additions & 0 deletions docs/source/how_tos/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
How-to Guides
#############

This page contains a list of How-tos that will help you get started with Picasso Workflow.

.. toctree::

configure_workflow
reuse_workflow
configure_docker_registries
trigger_build_with_cli
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Reusing the Picasso Workflow
###################################
Reuse Picasso Workflow in Your Repository
##########################################

This document will guide you through different ways of reusing the Picasso workflow.
This document will guide you through different ways of reusing the Picasso workflow in your own repository.

To use a reusable workflow from your own repository, you can use the ``uses`` keyword in your workflow file. The ``uses`` keyword specifies the location of the reusable workflow file, which can be a specific tag, branch, or commit, either in the same repository or a different repository.

Expand All @@ -11,6 +11,9 @@ For more information, see `Reusing workflows`_.

.. _`Reusing workflows`: https://docs.github.com/en/actions/sharing-automations/reusing-workflows

Workflow Dispatch Event
=======================

Here is an example of how to use the Picasso workflow using the `workflow_dispatch`_ event:

.. code-block:: yaml
Expand Down Expand Up @@ -61,6 +64,9 @@ Here is an example of how to use the Picasso workflow using the `workflow_dispat

In this example, you can trigger the workflow manually via the GitHub Actions tab, using the ``workflow_dispatch`` feature to input the necessary values. Based on the provided inputs, the workflow will then build the Open edX image.

Push Event
==========

Here's an example of how to use the Picasso workflow using the `push`_ event:

.. code-block:: yaml
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Trigger a build with the GitHub CLI
####################################
Trigger a New Build with the GitHub CLI
########################################

Consider you're using the Picasso Workflow like in the following snippet to build Open edX images:

Expand Down
10 changes: 4 additions & 6 deletions docs/README.rst → docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,8 @@ Here, you will find details on how to use and configure Picasso Workflow in your

.. _tutor-contrib-picasso: https://github.com/eduNEXT/tutor-contrib-picasso/

Contents
********
.. toctree::
:caption: Contents

- `Configurations <configurations.rst>`_
- `Workflow Usage <reuse_workflow.rst>`_
- `Configuring Docker Registries <configuring_docker_registries.rst>`_
- `Triggering a Build with the GitHub cli <trigger_build_with_cli.rst>`_
topic_guides/index
how_tos/index
8 changes: 8 additions & 0 deletions docs/source/topic_guides/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Topic Guides
#############

This page contains an overview of the key features of the Picasso Workflow.

.. toctree::

reusable_build_workflow
36 changes: 36 additions & 0 deletions docs/source/topic_guides/reusable_build_workflow.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
Reusable Build Workflow (Picasso)
##################################

The Picasso Workflow is a `reusable GitHub Actions workflow`_` designed to build Open edX Docker images for Tutor environments. It simplifies the process of building custom Open edX images with additional functionality, making it easier to maintain, extend, and deploy environments using modern CI/CD practices.
mariajgrimaldi marked this conversation as resolved.
Show resolved Hide resolved

Purpose
=======

Picasso is a tool designed to help teams simplify the build process for the Open edX Docker images, tailored explicitly for use in Tutor environments. It enables the addition of custom behaviors and features through the internal plugin `tutor-contrib-picasso`_, allowing for additional flexibility during the build process.

This GitHub Actions workflow replaces the existing `Jenkins-based pipeline`_ and integrates directly with other workflows, allowing it to be called in custom jobs. Picasso leverages Tutor to build images from the **Olive** version of Open edX onwards. The resulting Docker images can be used for both production and development environments, simplifying the process of managing multiple environments while ensuring consistency.

With Picasso, teams can build custom Open edX images with additional functionality, making it easier to maintain, extend, and deploy environments using modern CI/CD practices.

.. _`Jenkins-based pipeline`: https://github.com/eduNEXT/dedalo-scripts/blob/main/jenkins/picasso_v2
.. _reusable GitHub Actions workflow: https://docs.github.com/en/actions/sharing-automations/reusing-workflows

Workflow Overview
=================

The Picasso Workflow is designed to build Open edX Docker images for Tutor environments.

Key features of the Picasso Workflow include:

- **Runs on GitHub-hosted runners**: By default, the workflow uses ``ubuntu-latest`` runners to execute jobs. This can't be changed to self-hosted runners for the time being.
- **Builds and pushes Docker images**: The workflow pushes images to Dockerhub by default. This can be customized to push images to other registries.
- **Supports multiple services**: You can specify the service to build (e.g., ``openedx``, ``mfe``, ``codejail``, etc.) using the ``SERVICE`` input.
- **Customizable repository and strain**: The workflow allows for specifying the repository, branch, and path to the strain being built. This enables building images from different configurations.
- **Configurable BuildKit parallelism**: By default, the workflow limits parallelism during the build process to optimize resource usage, although this can be changed using the ``ENABLE_LIMIT_BUILDKIT_PARALLELISM`` input. This is useful for low-powered machines, like `Github Actions standard runners`_.
- **Private repository access**: SSH keys are used to clone private repositories securely. The SSH private key should be stored as a secret in the repository, and must have access to the repository specified in ``STRAIN_REPOSITORY``.
- **Extra commands**: The workflow allows running additional custom commands with ``tutor picasso run-extra-commands``. For details, refer to the `tutor-contrib-picasso`_ documentation.
- **Environment setup**: The workflow sets up and configures Tutor Virtual Environments (TVM), installs necessary plugins like ``tutor-contrib-picasso``, and prepares the environment to build and push Docker images using the `Tutor CLI`_.

.. _tutor-contrib-picasso: https://github.com/eduNEXT/tutor-contrib-picasso/
.. _Github Actions standard runners: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
.. _Tutor CLI: https://docs.tutor.edly.io/
15 changes: 15 additions & 0 deletions requirements/doc.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
sphinx==4.2.0
sphinx-book-theme==0.3.3
recommonmark==0.6.0
sphinxcontrib.images==0.9.4
sphinx_panels==0.6.0
sphinxcontrib.contentui==0.2.5
sphinx_copybutton==0.5.0
sphinxcontrib.mermaid==0.7.1
sphinxcontrib-applehelp==1.0.4
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-youtube==1.3.0
Loading