Development docs reorganization

[embray]

As discussed with @adrn and others on Slack, we would like to reorganize the Astropy developer docs.

The key points are to:

• Add a top-level landing page for development documentation. For this we take inspiration particularly from NumPy's development docs: https://numpy.org/doc/stable/dev/index.html (I would not mind even going so far as to borrow some of their text almost verbatim). It would feature a clean and direct guide to getting started on Astropy (much less jumbled than this page.

• At the bottom of the top-level page, add a table of contents for sub-pages that contain more detailed discussions. Most of those would be the same as the existing development pages, though some of them might merit additional reorganization/rewriting.

• I like some aspects of the SciPy developer docs as well, especially the "ways to contribute" section: http://scipy.github.io/devdocs/dev/index.html

• The matplotlib documentation is also a good inspiration. In particular we like the "choose your own adventure" style with prominent links for where to get started for different types of contributions.

• Maybe rename from "Developer documentation" to "Contributor documentation" or "Contributor guide"?

[pllim]

Maybe we can also emulate other guides out there?

• PlasmaPy (https://docs.plasmapy.org/en/stable/contributing/index.html) cc @namurphy
• OpenAstronomy (https://packaging-guide.openastronomy.org/en/latest/) cc @Cadair et al.
• Scientific Python (https://learn.scientific-python.org/development/guides/) cc @jarrodmillman et al.
• pyOpenSci is trying to make one at https://www.pyopensci.org/python-package-guide/CONTRIBUTING.html but it looks incomplete, but we should keep an eye on it in case it is nicely populated by the time we work on this issue, cc @lwasser

[Cadair]

Also sunpy's dev guide: https://docs.sunpy.org/en/latest/dev_guide/index.html

[lwasser]

This is not a conversation i think for this issue - but i wanted to suggest an idea
The guide on the scientific python website has a lot of great information. it was the scikit-hep communities guide.
for us at pyOpenSci it's important that all of our content is written via community consensus and is also accessible to those who are newer to packaging. and of course it supports peer review so it has a different angle.

i also think some of these guides also have slightly different angles - contributing vs development vs packaging which are all related but have subtle differences and audiences.

• @namurphy @Cadair et al what do you think about working together on a single guide with pyOpenSci? This might be a situation where we use information from existing guides and link back to them but also implement our normal content review process. or we just all join forces.

our ecosystem is filled with various guides. can you imagine what we could do if we all just worked together on making one great one that we all keep updated but that's also driven but content developed via community consensus?

it's just a thought.

A baby step could be that we think about this existing content as we develop more of our guide (we are currently fleshing out an outline!). We could work with you to link back and borrow text with full credit going to each author of the existing guides. But the content would then then goes through community review. if you are interested in / open to this I could open a new issue in our packaging guide repo about it.

python packaging can be overwhelming for many because of all of the options. maybe we can work together to consolidate some of those options in a way that still allows each amazing community mentioned to do the great work you all are doing :)

[namurphy]

our ecosystem is filled with various guides. can you imagine what we could do if we all just worked together on making one great one that we all keep updated but that's also driven but content developed via community consensus?

I have had the same thought fairly often! My perspective is that it would be great but difficult to have a shared guide. PlasmaPy's contributor guide is a mix of very general stuff (like the code contribution workflow) and rather specific stuff (like how plasma physicists use the electron-volt as a unit of temperature grumble grumble).

A big part of the difficulty is that the packaging practices between different packages are quite different...probably because the Python packaging landscape is quite fragmented (as you pointed out). For a shared guide, the participating packages would need to have a shared set of packaging practices...and contributors may be stretched too thin and/or burnt out to have the capacity to work towards that...even though in the long run it would be less work for all. Each package probably also has a unique set of tooling (i.e. pre-commit hooks, etc.) so I'm wondering if our community's guides can be made modular.

In any case, I'd be happy to be part of these conversations if you create an issue about it in the packaging guide repo, and could potentially help with some content on writing clean scientific software. Thank you for bringing this up!

[Cadair]

I think it's probably important to differentiate between "packaging guides" and "developer documentation" here, a packaging guide probably has a lot more that can be shared, and I think that's why we have the PyOpenSci and OpenAstronomy ones (they are both shared knowledge with different scopes, openastronomy being much more opinionated to what astropy and sunpy do at the moment).

For developer documentation, it's a little trickier as @namurphy says, some of it is definitely common over packages, but a lot of it isn't. For example even Astropy and SunPy (two projects which are very similar) have subtly different developer conventions and workflows. I think some large parts of SunPy's Newcomers guide: https://docs.sunpy.org/en/latest/dev_guide/contents/newcomers.html could be shared between a larger set of packages (we already link out to the astropy documentation).

[lwasser]

absolutely. @Cadair @namurphy to avoid distracting @pllim effort here for astropy docs - how about this. i'll do a review of all of the docs we have listed here and will try to identify different types of content that are overlapping, could be included in pyos content, needs to be community specific, etc. I just opened an issue here in our discourse about this. i'll try to identify parts of each of the guides above that are general enough to be in a single guide that we (could) all contribute to - no pressure!! I'll also ID things that are community specific. Feel free to join and follow along there (github login makes it easy). The goals can then be:

• over time perhaps we move towards a bit more consistent dev practices (emphasis on over time) and infrastructure.
• To avoid placing undue burden on your communities to try to merge or redo anything (that is a lot of work). Rather let pyOpenSci be the support system - we can evaluate options and present them for vetting and decide from there. We are working on a guide anyway so this fits nicely with our existing goals.

A baby step to begin with can be - once the inventory is done (i'll do it). We can try to unify what's out there and cross link, share content, etc which you are already doing.

[pllim]

Let's move the template discussions for Astropy over to here:

• Scope out how we want to template using existing tools and what it gives the community astropy-project#338

Thanks, everyone!