With @rouk1 we looked at this for a bit with our limited knowledge of sphinx/rst.
We also looked into how pandas and sklearn achieve this.

Here are the two ideas we had:

Change file structure of the generated/ directory

sphinx-pydata-theme knows to add dropdowns for nested pages as shown in this example, but our API docs are all laid out flatly in the generated/ directory. In sklearn, a special script is used to enforce a nested structure. We don't know how we could achieve this, and the accessor mechanic seems to make it yet more complicated.

Rely on show_nav_level option

As detailed here, the show_nav_level setting can be used to collapse sections that have a :caption: (sections without a caption are hidden). However we can't get the hierarchy of headings right yet. This is the most promising idea IMO.

_{ps: sphinx is hard 😢}

From a quick look at it, the thing I would try ¹ is to try to mirror pandas structure:

• they have some kind of index for the reference API that defines a toctree with sections https://github.com/pandas-dev/pandas/blob/main/doc/source/reference/index.rst
• each section has its own file with autosummary https://github.com/pandas-dev/pandas/blob/main/doc/source/reference/groupby.rst

This may be possible to do it in a single file, but I have to admit I never managed to understand the sphinx toctree and how this relates to the left-hand-side navbar ...

@glemaitre Maybe you have an idea? The accessor documentation logic makes this harder it seems. I'd be happy to pair on this