Docs Round 1

[willtebbutt]

This is the first round of documentation. The plan is to explain Tapir.jl's rule abstraction, and the tangent type system. The audience is intended to be those who are potentially interested in developin Tapir, rather than just using it.

This is not ready for review yet.

Note: if reading, be aware that I'm making extensive use of docstrings to create the documentation as this reduces the amount of duplication of documentation, and makes it easy to use doctests to avoid letting the documentation going out-of-sync with the code in future. Therefore, you should either use the docs preview, or build it locally, to get a proper sense of what the docs look like.

edit: todo

• improve explanation of derivatives
• improve X^T X explanation
• improve first example involving mutation
• improve citations
• provide generalised explanation of rrule interface

[github-actions]

Performance Ratio:
Warning: results are very approximate!

┌────────────────────────────┬────────┬─────────┬─────────────┬─────────┐
│                      Label │  Tapir │  Zygote │ ReverseDiff │  Enzyme │
│                     String │ String │  String │      String │  String │
├────────────────────────────┼────────┼─────────┼─────────────┼─────────┤
│                        sum │   42.8 │   0.463 │        3.01 │   0.638 │
│                       _sum │   6.77 │   528.0 │        26.9 │   0.118 │
│                   kron_sum │   81.7 │    3.43 │       216.0 │    26.1 │
│              kron_view_sum │   99.2 │    11.5 │       248.0 │    9.38 │
│      naive_map_sin_cos_exp │   4.14 │ missing │        8.62 │    2.82 │
│            map_sin_cos_exp │   4.79 │    1.86 │        7.53 │    3.41 │
│      broadcast_sin_cos_exp │   4.68 │    2.61 │        1.65 │    2.82 │
│                 simple_mlp │   8.82 │    3.02 │        11.2 │    3.07 │
│                     gp_lml │   15.5 │    4.34 │     missing │ missing │
│ turing_broadcast_benchmark │   8.73 │ missing │        26.2 │ missing │
└────────────────────────────┴────────┴─────────┴─────────────┴─────────┘

[mhauru]

I read through the Intro and Algorithmic Differentiation pages, and took a peek at the Mathematical Interpretation of Functions one as well. Overall, this is very clearly written, and helped me a lot. I have essentially no previous experience of AD implementations and how they work.

I left a few comments. Related to the first comment, I think my largest confusion was with the infinitesimals/differentials \text{d} x. The text also at a couple of points switched between that and \dot{x}. They seem to be considered as elements of the same set as x itself, which I found confusing given they are infinitesimal and more like tangents. I'm not sure what the best thing to do here is, but given that the later section about implementation seems to have to introduce the notion of tangent types anyway, I wonder if it would be useful to do so here in mathsland already. If not that, then a note about how the reader should think of \text{d} x/\dot{x} would be good.

[torfjelde]

Had a skim through it all and it looks very nice:)

I don't really have any concrete feedback beyond what @mhauru has already given. I was also a bit confused by the switch between $d x$ and $\dot{x}$, in part because it eludes to the fact that you should actually define the gradient as the time-derivative for a particular curve at that given point but this is (AFAIK) not mentioned anywhere. Buuut those who know this can probably can probably handle this minor thing, while those who don't know this won't ofc have this moment of "so is this the time derivative or is it something else?" 🙃

But all in all, I definitively think the docs achieve what it sets out to do! Many of the things you've mentioned to me about Tapir.jl before now makes more sense and I understand some of the design choices made much better:)