docs: improve website organization

[tomMoral]

Related to doc improvements described in #1008

Summary of the changes

• remove GIF and replace by executable code snippet
• reorganized landing page with clickable boxes with links to motivation, tutorials, installation, and citation
• update appearance of API reference to be less cluttered
• remove tutorial dropdown in the navigation bar

New docs workflow with mike

• mike allows us to have different docs versions in parallel

• we want to have one dev version showing the docs of the main branch (unreleased), and one latest version showing the docs of the latest release

• the website docs should always show the latest version

• whenever we push to main, the dev version is updated:

  • tutorials and examples notebooks are converted into markdown and the docs files are created using mkdocs build

• whenever we publish a new release, the same thing happens, but the latest docs is updated (-u option in the command) to the new release and deployed on the website.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.27%. Comparing base (2fd89a8) to head (988afa4).
Report is 6 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1147      +/-   ##
==========================================
- Coverage   84.43%   76.27%   -8.16%     
==========================================
  Files          97       97              
  Lines        7673     7677       +4     
==========================================
- Hits         6479     5856     -623     
- Misses       1194     1821     +627     

Flag	Coverage Δ
unittests	76.27% <ø> (-8.16%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 27 files with indirect coverage changes

[janfb]

Note: for building the docs site locally I needed to run the following commands in an empty condo env with python=3.10:

pip install -e ".[doc,dev]"
pip install jupyter_contrib_nbextensions
pip install --upgrade notebook==6.4.12
pip uninstall traitlets 
pip install traitlets==5.9.0
pip uninstall ipython
pip install ipython==8.9.0

...

mkdocs serve

I believe this is due to our custom Jupyter notebook extensions (e.g., jupyter_contrib_nbextensions) not being compatible with the new notebook 7 release from last year. As a consequence, we need to downgrade notebook, traitlets and ipython.

[janfb]

this is great, thanks a lot!

I added some comments and pushed 2 commits implementing some of my suggestions.

[tomMoral]

A few more comment, I will take another look at the mkdocs-jupyter extension in the afternoon

[janfb]

@tomMoral I made some edits here:

• shorter code snippet
• moved list publications / implemented algorithms back to home page
• remove getting started at the bottom of home
• use separate index.md for tutorials and API refs and remove the dropdown menu in the navigation bar

What do think of those?

And I have a couple of comments / questions:

• at the moment it seems we have a copy of all notebooks in docs/tutorials and docs/examples and we convert them to markdown at build time. (We also have a tutorial and a tutorials folder, but I think this is an artifact of the old release workflow)
• I think this is not ideal: if we change the actual notebooks at the root level we need to update those as well, no?
• also, as you mentioned already, it takes about 10s at every build.
• couldn't we convert them to markdown from the root level directly and do that with a separate command only when we deploy the website (like it was done before: https://github.com/sbi-dev/sbi/wiki/Release-workflow#update-the-documentation-to-the-new-version
• and then we could wrap all the required commands into a GitHub action to have a release workflow?

We could also arrange a video call to discuss.

[tomMoral]

I finally got some time to take a look at this one.
I like the updates and I think the new look is more accessible.

I just pushed 2 small changes, one adding a link to "get started" tutorial on the front page. Feel free to revert if you don't like it.

As for mkdocs-jupyter:

• The docs/tutorials and docs/examples are symlink to the root level ones so they are the same :)
• but indeed 10s is slow for serving.

This is really as you prefer for the workflow.
I would tend to say that having a fast iteration path for updating the doc is better so I will try and see if we can automatize calling nbconvert.

[janfb]

Hi @tomMoral

thanks for the update! It looks good.

I made a couple of smaller changes:

• edited the landing page: updated code snippet, updated implemented methods
• merge main into this branch
• update tutorials of implemented methods
• update credits

Two points remain:

1. It seems when clicking on the tutorial notebooks and looking at them on the website in markdown, they do not show the code but only the markdown content and the output. See for example "Getting started". So there seems to be something off here.
2. Workflow: I think it would be good if we have GH workflow that is launched whenever me make a new release, i.e., not on every push to main. The reason is that we want to keep the docs in sync with the pypi version of sbi, not with main. I would also prefer to separate the notebook conversion from the website building. In other words, we include a manual call to nbconvert into the workflow that deploys new docs upon a new release, but when I work on the website locally it not re-convert the notebooks with every change to the website. Does that make sense?

[janfb]

@tomMoral In the last commit I made a couple of naive changes to the workflow file, all up for discussion:

• action is issued upon a published release
• converts notebooks from the source dir into the docs/ into markdown, just like we did manually before
• directly deploy the docs onto the website.

Does that make sense?

[tomMoral]

• I removed mkdocs-jupyter, which we are not using anymore.
• For the doc versioning, it seems that mike is a package to have multiple versions (one dev and one per release, with a latest alias which would be the default).
• I updated the workflow to use it for now.

[janfb]

TODO for setting up mike and rooting it on our "latest" docs version 0.22:

• make local backup of docs folder
• delete all previous docs using `mike delete --all
• checkout on tag of the last release v0.22.0
• remove old and rebuild tutorials and examples into markdown.
• build the docs using mike deploy 0.22.0 latest -u --push
• set default docs using mike set-default latest
• update Wiki release workflow: https://github.com/sbi-dev/sbi/wiki/Release-workflow

[janfb]

@michaeldeistler can you please give this a final review? To do so

• checkout locally and install the new docs dependencies: pip install -e ".[doc]"
• follow instructions in contribute.md for building docs locally, i.e., cd into docs and run the jupyter nbconvert commands.
• run mike serve in docs/
• see website on localhost.

See also PR description on top which summarizes all the changes.