notebooks in CI (again ;) - Jupyter kernel name; remove output; rendering?

[hagrid67]

Objectives
Establish some standards for editing notebooks in order to:

• avoid a "missing kernel" error in CI (see context below)
• avoid storing output to make code changes / diffs easier to track?
  • VSCode allows filtering of output differences (and metadata) so perhaps it's OK to commit outputs
• enable rendering in github - seems to require output to be committed🤦‍♂️

(This could just be a standards / documentation task)

Todos

• identify how to avoid the missing kernel error
• document using nbconvert --clear-output --inplace notebook.ipynb prior to add+commit
• try to ensure that notebooks are rendered in github views
  • eg generate notebooks containing output in the CI process and store them in a rendered-notebooks folder?

Minimal testing requirements
I think this is a change / addition to coding standards rather than a code change.

Additional context

The default jupyter kernel seems to be called "python3" in Jupyter; eg try running the following command from a local venv with Jupyter:
jupyter kernelspec list

I have several venvs which I use for Flatland and other things. I tend to call them something meaningful like ve310fl. In Jupyter I configure a kernel to use the given venv. Sometimes I use jupyter from a different venv, eg Jupyter in venv ve38torch might have a kernel pointing at ve310fl; I give it the same name, ie ve310fl. When editing a notebook I may switch the kernel from python3 to ve310fl. But the jupyter installed in CI contains only the default kernelspec python3 so running the notebook fails because ve310fl is missing.

    graph
    subgraph local_ve["local Virtual env"]
        kernel_ve310fl[Kernel ve310fl]  -- "Jupyter\nlocal config" --> ve310fl[Venv ve310fl bin/python3]
        notebook[my_notebook.ipynb] -- "kernelspec\n've310fl'\ninstead of 'python3'" --> kernel_ve310fl
    end

    subgraph CI_ve["CI python env / venv?"]
        kernel_py310["Kernel 'python3'"]  -- "Jupyter\ndefault kernel" --> py310[Python 3 container?\n /bin/python3]
        notebook_CI[my_notebook.ipynb] -- "kernelspec\n've310fl'\ninstead of 'python3'" --> 1["Error: Missing Kernelspec"]
    end

    local_ve -- Github commit + push --> CI_ve

%% defining styles
    classDef Red fill:#AA4444;
%% assigning styles
    class 1 Red;

Loading

Some info about how Jupyter defines its default kernel here