Provide tutorial as a notebook

[szhorvat]

It would be nice to provide the tutorial as a Jupyter notebook, so people can directly evaluate code without having to copy it first. Is this easy to do @iosonofabio ?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[ntamas]

Relevant docs from Readthedocs re Jupyter integration are here.

Personally, I'd be in favour of converting the tutorial to Jupyter notebooks, but I would not try to maintain two versions in parallel (a Jupyter notebook and a non-interactive version).

Jupytext seems to be a good candidate for managing the source of the notebook in a VCS-friendly format and then we can generate the notebook from it at build time. Alternatively, there are many Markdown variants that we could use. If the tutorial is code-heavy, Jupytext is a better option, otherwise we could use Markdown.

[iosonofabio]

I can give this a go. I'll aim to open a PR against develop, right?

[szhorvat]

No reason this can't go in the 0.10.x series IMO. Let's try not to put things unnecessarily on develop, which then will create pressure for a premature 0.11 C/igraph release ...

[iosonofabio]

I tried playing with it a little bit. The easiest would be to convert the tutorial.rst into a .py file and use sphinx-gallery again, but this time for the long tutorial. Otherwise, using jupytext to convert md to ipynb and then gong from there to rst seems to have a lot of conflicts.

The tutorial has several changes in develop that I don't want to rewrite, so if I'm doing this, I'll do it against develop (sorry @szhorvat). Pressure on 0.11 is good, the mpl rewrite is a big thing and adding tutorials on a deprecated API is not really worth it, which is a big reason why I'm slowing down my contributions at the moment.

[szhorvat]

Why does a script that converts the tutorial to a notebook depend on the contents of the tutorial?

Pressure on 0.11 is good

If you want 0.11 earlier, then you can help out with implementing the necessary changes on the C side.

[szhorvat]

I think it would be best to decouple major python igraph releases from C ones.

Why not release python igraph 0.11 with the mpl changes (which are important) based on the current C igraph 0.10?

This would give everyone the most flexibility.

Opinions?

[iosonofabio]

I'd just like to say, I am becoming quite busy with my lab so reimplementing things twice is not really feasible. If you want to better understand why that is affected by the mpl change, please take a look at sphinx gallery and how we use it in our docs.

I'm also happy to wait until develop has been released, it's not urgent anyway.

Thank you, unfortunately I really have too little time to help with the C core at the moment.

[szhorvat]

Why not release python igraph 0.11 with the mpl changes (which are important) based on the current C igraph 0.10?

Any comments on this @ntamas and @iosonofabio ?

I understand @iosonofabio's concerns and motivation for wanting the next release soon and I agree with him. I think this would be a good solution with virtually no drawback (other than a bit of a version confusion for some people) that would give us the flexibility we need.

Trying to keep python-igraph and C/igraph in lockstep will always be inconvenient for either one side or the other. It is now the second time that python-igraph accumulated C-independent breaking changes that would be very useful (to users) to release, not to mention that not releasing them hinders development.

[ntamas]

For the record: we have discussed this with @szhorvat online and the decision was to move forward with python-igraph 0.11 in a way that it is still based on version 0.10.x of the C core -- there is no need to keep the two version numbers in sync.

[iosonofabio]

That's great, I agree we might have been a little uptight on that issue given that rigraph is already decoupled.

I've made a draft PR exploring the use of sphinx-gallery(which we already use) to make a jupyter notebook of the tutorial. It's kind of difficult, mainly because:

• cross-references are difficult to track
• code blocks inside of e.g. .. note or .. warning blocks are quite undefined in notebooks... should they be a cell, or not?
• sphinx-gallery requires the entire tutorial to be reformatted as a literate .py file (using # %% to mark non-code cells), which is kind of a pain - I've done 200 lines or so.

I've also looked into jupytext, which converts notebooks into markdown/rst. The issues are similar, cross-referencing is poor in notebooks, and note/warning/expected failures are difficult too, and a straightout conversion of the .rst into a notebook would be needed.

I've also found https://github.com/QuantEcon/sphinxcontrib-jupyter which is one step closer to what we want, i.e. conversion of an existing rst file into a notebook without having to rewrite the whole thing. The issue there is that it's unmaintained since 2020. Nonetheless, the backing organisation (https://quantecon.org/) is alive and kicking, co-run by US and Aussie folks, and under the numfocus umbrella. It would be conceivable to link up with them and see how much they care about that repo.

Any comments?

[ntamas]

code blocks inside of e.g. .. note or .. warning blocks are quite undefined in notebooks... should they be a cell, or not?

I guess that the easiest solution would be to avoid using code blocks in .. note or .. warning blocks? After all, if the text is long enough and includes some code to warrant an additional code block, then maybe it should be part of the main text.

The cross-referencing stuff seems to be a pain to deal with, but we will need cross-referencing to make stuff user-friendly, so I'd rather let go the idea of converting the tutorial to a notebook if it causes so many complications down the road.

It would be great if you could reach out to QuantEcon to see if the extension is maintained or whether it works with recent Sphinx versions at all. There were a few API changes in Sphinx 7 that might cause problems.