docs, dev-requirements: upgrade sphinx and try to fix some type issues with documentation

[Exirel]

Description

First, I upgraded Sphinx to the latest version, which requires to upgrade Furo as well. That went well so that's good! Then I added links to the general index and the Python module index, because they are nice.

Then I tried to fix Sphinx autodoc's warnings regarding type annotations. I more or less figure out a way to fix one or two... but then I got stuck very hard: Sphinx, currently, does not handle type annotation properly. For some reason it works for TypedRule now but not with some datetime.datetime annotation? There are clearly bugs (that are not new to version 7.0 of Sphinx by the way), and I tried few options (such as using autodoc_type_aliases, or for a moment I switch to another autodoc extension, with the same issues). Sopel is not an isolated case, as this issue shows it.

So yeah, since it's mostly Sphinx's fault, I decided to give up on the remaining warnings. I don't think we can do much: we could, hypothetically, tinker for hours with the code... but I don't see the value here. I already spent quite a few hours on this and I'm not happy so let's not waste any more time.

Note: this won't work with Python 3.7 as I use a feature from Python 3.8 (typing.Protocol). So until @dgw publish his PR to remove Python 3.7 support, this will be a draft only.

Checklist

• I have read CONTRIBUTING.md
• I can and do license this contribution under the EFLv2
• No issues are reported by make qa (runs make quality and make test)
• I have tested the functionality of the things this change touches

[dgw]

Line notes are self-explanatory. Besides a CodeQL warning about SopelIdentifierMemory not overriding __eq__() (not actually new, but has been triggered nonetheless by touching that class's code), I thought it would be useful to point out the Sphinx warnings that are still present in the current version of this PR, courtesy of Netlify's last build log.

Click for Sphinx warnings

4:56:47 AM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage:1: WARNING: py:class reference target not found: ModeDetails
4:56:47 AM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage:1: WARNING: py:class reference target not found: PrivilegeDetails
4:56:47 AM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage:1: WARNING: py:class reference target not found: ModeTuple
4:56:47 AM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage.ignored_modes:1: WARNING: py:class reference target not found: ModeTuple
4:56:47 AM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage.modes:1: WARNING: py:class reference target not found: ModeDetails
4:56:47 AM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage.privileges:1: WARNING: py:class reference target not found: PrivilegeDetails
4:56:47 AM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.parse_modestring:1: WARNING: py:class reference target not found: ModeTuple
4:56:47 AM: /opt/build/repo/sopel/plugins/handlers.py:docstring of sopel.plugins.handlers.AbstractPluginHandler.get_capability_requests:1: WARNING: py:class reference target not found: plugin_decorators.capability
4:56:47 AM: /opt/build/repo/sopel/plugins/handlers.py:docstring of sopel.plugins.handlers.PyModulePlugin.get_capability_requests:1: WARNING: py:class reference target not found: plugin_decorators.capability
4:56:47 AM: /opt/build/repo/sopel/plugins/rules.py:docstring of sopel.plugins.rules.AbstractRule.is_channel_rate_limited:1: WARNING: py:class reference target not found: datetime
4:56:47 AM: /opt/build/repo/sopel/plugins/rules.py:docstring of sopel.plugins.rules.AbstractRule.is_user_rate_limited:1: WARNING: py:class reference target not found: datetime
4:56:47 AM: /opt/build/repo/sopel/plugins/rules.py:docstring of sopel.plugins.rules.Rule.is_channel_rate_limited:1: WARNING: py:class reference target not found: datetime
4:56:47 AM: /opt/build/repo/sopel/plugins/rules.py:docstring of sopel.plugins.rules.Rule.is_user_rate_limited:1: WARNING: py:class reference target not found: datetime
4:56:47 AM: /opt/build/repo/sopel/tools/identifiers.py:docstring of sopel.tools.identifiers.Identifier.casemapping:1: WARNING: py:class reference target not found: sopel.tools.identifiers.Casemapping
4:56:47 AM: /opt/build/repo/sopel/tools/memories.py:docstring of sopel.tools.memories.SopelIdentifierMemory.make_identifier:1: WARNING: py:class reference target not found: sopel.tools.identifiers.IdentifierFactory
4:56:47 AM: /opt/build/repo/sopel/tools/target.py:docstring of sopel.tools.target.Channel.make_identifier:1: WARNING: py:class reference target not found: IdentifierFactory
4:56:47 AM: /opt/build/repo/sopel/trigger.py:docstring of sopel.trigger.Trigger:1: WARNING: py:class reference target not found: Match

Since I'm unsure how long Netlify keeps build logs around, or whether the deploy preview logs are public, I also pasted the relevant portion here.

[dgw]

Was looking for something else (after someone asked a question in Furo's discussions) and noticed https://deploy-preview-2496--sopel-unstable-docs.netlify.app/package/db#sopel.db.SopelDB has a not-very-helpful annotation of a parameter:

(It's the <class 'sopel.tools.identifiers.Identifier'> notation. It's too verbose, and it's not even x-ref'd.)

Along the way to that, I also noticed some long signatures and remembered about sphinx-doc/sphinx#11011 adding a configuration setting (maximum_signature_line_length) in Sphinx 7.1.0 that makes long signatures multi-line, just like we do in the code (and Furo should handle it fine). Want to make that part of this patch, since you're upgrading Sphinx anyway?

[dgw]

Uh-oh. We'd better keep an eye on this. "Furo is broken on Sphinx 7.2" pradyunsg/furo#693

[dgw]

Quick responses! pradyunsg/furo#693 (comment) (Furo fix) + pradyunsg/furo#693 (reply in thread) (Sphinx fix)

[Exirel]

Along the way to that, I also noticed some long signatures and remembered about sphinx-doc/sphinx#11011 adding a configuration setting (maximum_signature_line_length) in Sphinx 7.1.0 that makes long signatures multi-line, just like we do in the code (and Furo should handle it fine). Want to make that part of this patch, since you're upgrading Sphinx anyway?

Hell yeah, and it works!

[dgw]

Hmm, I wonder why solving the Sphinx requirement failed… There's a new Furo version (2023.9.10) you can use as an excuse to re-run CI 😁

[SnoopJ]

Hmm, I wonder why solving the Sphinx requirement failed…

I believe it's because the relevant requirement is sphinx<8,>=7.2.3, but Sphinx (surprisingly?) dropped 3.8 support in 7.2.0

[dgw]

surprisingly?

A bit. 3.8 isn't EOL until over a year from now (October '24), and it's also weird (to me) to drop a Python version in a minor bump.

That said, SemVer is meaningless… but oy. I guess we can probably get away with Sphinx 7.2.x for Python >=3.9 and Sphinx 7.1.x for Python 3.8. We Probably™ don't rely on any of the fixes/changes in Sphinx 7.2.0, and Furo should have finally sorted out all of its cross-version compatibility at this point.

I hope. 😅

[dgw]

@Exirel I looked at the changes since my last review and I think this is fine except for the failure on Python 3.8. Could you sort that out when you get a moment, so we can verify that the docs build correctly? Looks like Netlify didn't even try to make a deploy preview for the latest commit; there's no check annotation. ☹️

[dgw]

I think this qualifies as nitpicking now. Just force-push it; as long as you don't also rebase, I'll be able to use the "force-pushed" event to view what changed for re-review purposes. 😸

[dgw]

Before I re-review this one last time and give you the squash instruction, are the Sphinx warnings in the build log for cdc9a10 part of what you were trying to fix?

Long Sphinx log snippet

3:37:31 PM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage:1: WARNING: py:class reference target not found: ModeDetails
3:37:31 PM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage:1: WARNING: py:class reference target not found: PrivilegeDetails
3:37:31 PM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage:1: WARNING: py:class reference target not found: ModeTuple
3:37:31 PM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage.ignored_modes:1: WARNING: py:class reference target not found: ModeTuple
3:37:31 PM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage.modes:1: WARNING: py:class reference target not found: ModeDetails
3:37:31 PM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.ModeMessage.privileges:1: WARNING: py:class reference target not found: PrivilegeDetails
3:37:31 PM: /opt/build/repo/sopel/irc/modes.py:docstring of sopel.irc.modes.parse_modestring:1: WARNING: py:class reference target not found: ModeTuple
3:37:31 PM: /opt/build/repo/sopel/plugins/handlers.py:docstring of sopel.plugins.handlers.AbstractPluginHandler.get_capability_requests:1: WARNING: py:class reference target not found: plugin_decorators.capability
3:37:31 PM: /opt/build/repo/sopel/plugins/handlers.py:docstring of sopel.plugins.handlers.PyModulePlugin.get_capability_requests:1: WARNING: py:class reference target not found: plugin_decorators.capability
3:37:31 PM: /opt/build/repo/sopel/plugins/rules.py:docstring of sopel.plugins.rules.AbstractRule.is_channel_rate_limited:1: WARNING: py:class reference target not found: datetime
3:37:31 PM: /opt/build/repo/sopel/plugins/rules.py:docstring of sopel.plugins.rules.AbstractRule.is_user_rate_limited:1: WARNING: py:class reference target not found: datetime
3:37:31 PM: /opt/build/repo/sopel/plugins/rules.py:docstring of sopel.plugins.rules.Rule.is_channel_rate_limited:1: WARNING: py:class reference target not found: datetime
3:37:31 PM: /opt/build/repo/sopel/plugins/rules.py:docstring of sopel.plugins.rules.Rule.is_user_rate_limited:1: WARNING: py:class reference target not found: datetime
3:37:31 PM: /opt/build/repo/sopel/tools/identifiers.py:docstring of sopel.tools.identifiers.Identifier.casemapping:1: WARNING: py:class reference target not found: sopel.tools.identifiers.Casemapping
3:37:31 PM: /opt/build/repo/sopel/tools/memories.py:docstring of sopel.tools.memories.SopelIdentifierMemory.make_identifier:1: WARNING: py:class reference target not found: sopel.tools.identifiers.IdentifierFactory
3:37:31 PM: /opt/build/repo/sopel/tools/target.py:docstring of sopel.tools.target.Channel.make_identifier:1: WARNING: py:class reference target not found: IdentifierFactory
3:37:31 PM: /opt/build/repo/sopel/trigger.py:docstring of sopel.trigger.Trigger:1: WARNING: py:class reference target not found: Match

I ask because at least some of the reference targets not found are part of your diff, as aliases in docs/source/conf.py.

[Exirel]

Yeah, I tried, but as I said in the PR's description, for some reasons Sphinx just can't handle some cases, and I still don't understand why or how. I gave up on these. 😭

[Exirel]

All squashed and ready to go.

[dgw]

I've just looked over all the changes again, finally. (Made a good break from esoteric stuff like #2525, not gonna lie.) This patch has been in-flight more than long enough already!

While there were a couple of little wording changes I would otherwise ask to make, it's on me for missing them through all the previous reviews. Little documentation tweaks are great… to ignore until it's time to start working on 8.0.1.