Docs Round 1 (#175)

* Initial work

* Use Tapir in docstrings

* Some more writing

* Some more work

* More work

* Proper referencing, better jacobians, and general tidy up

* Add refs

* Do not run doctests during regular CI

* Remove documenter load in regular tests

* More tidying up

* Better stucture

* State rrule post-conditions

* Some tidying

* Fix typos in tangents docstring

* More improvements

* Tweak gradient wordin

* Update deployment location

* Fix some typos

* Fix more typos

* Tweak formatting and writing

* Add doctest to value_and_gradient

* Add doctest to __value_and_gradient

* More work

* Fix typo

* Remove redundant sentence

* Update docs/src/algorithmic_differentiation.md

Co-authored-by: Markus Hauru <[email protected]>

* Tidy up derivation

* Fix set notation

* Respond to Tor and Markus' comments

* Clarify notation further

* Fix typo in readme

* Restrict to at least Julia 1-10

* Rename a page

* Fix typo in fdata docstring

* Add rule docs

* Clarify adjoint notiation

* Use usual derivative notiation

* Fix some missed cases of alternative notiation

* Clarify dot notiation

* Remove duplicate docs

* Improve interface docstrings

* Document known limitations

* Improve safe mode documentation

* Initial safe mode documentation

* Improve safe mode documentation

* Tidy up a bit

* Improve safe mode docs

---------

Co-authored-by: Markus Hauru <[email protected]>

Project.toml

@@ -49,7 +49,7 @@ SpecialFunctions = "2"
 StableRNGs = "1"
 TemporalGPs = "0.6"
 Turing = "0.32"
-julia = "1"
+julia = "1.10"
 
 [extras]
 AbstractGPs = "99985d1d-32ba-4be9-9821-2ec096f28918"

README.md

@@ -142,7 +142,7 @@ Optimising to get similar performance to `Enzyme.jl` is an on-going process.
 
 ### What things should work well
 
-Noteworthy things which should be work and be performant include:
+Noteworthy things which should work and be performant include:
 1. code containing control flow
 1. value-dependent control flow
 1. mutation of arrays and mutable structs

docs/Project.toml

@@ -1,5 +1,7 @@
 [deps]
+ADTypes = "47edcb42-4c32-4615-8424-f2b9edc5f35b"
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
+DocumenterCitations = "daee34ce-89f3-4625-b898-19384cb65244"
 Tapir = "07d77754-e150-4737-8c94-cd238a1fb45b"
 
 [compat]

docs/make.jl

@@ -1,10 +1,40 @@
-using Documenter, Tapir
+using Documenter, DocumenterCitations, Tapir
+
+DocMeta.setdocmeta!(
+    Tapir,
+    :DocTestSetup,
+    quote
+        using Tapir
+    end;
+    recursive=true,
+)
 
 makedocs(
     sitename="Tapir.jl",
-    format=Documenter.HTML(),
+    format=Documenter.HTML(;
+        mathengine = Documenter.KaTeX(
+            Dict(
+                :macros => Dict(
+                    "\\RR" => "\\mathbb{R}",
+                ),
+            )
+        ),
+    ),
     modules=[Tapir],
     checkdocs=:none,
+    plugins=[
+        CitationBibliography(joinpath(@__DIR__, "src", "refs.bib"); style=:numeric),
+    ],
+    pages = [
+        "Tapir.jl" => "index.md",
+        "Understanding Tapir.jl" => [
+            "Introduction" => "understanding_intro.md",
+            "Algorithmic Differentiation" => "algorithmic_differentiation.md",
+            "Tapir.jl's Rule System" => "mathematical_interpretation.md",
+        ],
+        "Known Limitations" => "known_limitations.md",
+        "Safe Mode" => "safe_mode.md",
+    ]
 )
 
-deploydocs(repo="github.com/withbayes/Tapir.jl.git", push_preview=true)
+deploydocs(repo="github.com/compintell/Tapir.jl.git", push_preview=true)

docs/src/index.md

@@ -1,3 +1,11 @@
 # Tapir.jl
 
 Documentation for Tapir.jl is on it's way!
+
+Note 02/07/2024: The first round of documentation has arrived.
+This is largely targetted at those who are interested in contributing to Tapir.jl -- you can find this work in the "Understanding Tapir.jl" section of the docs.
+There is more to to do, but it should be sufficient to understand how AD works in principle, and the core abstractions underlying Tapir.jl.
+
+Note (29/05/2024): I (Will) am currently actively working on the documentation.
+It will be merged in chunks over the next month or so as good first drafts of sections are completed.
+Please don't be alarmed that not all of it is here!

docs/src/known_limitations.md

@@ -0,0 +1,128 @@
+# Known Limitations
+
+Tapir.jl has a number of known qualitative limitations, which we document here.
+
+## Circular References
+
+To a large extent, Tapir.jl does not presently support circular references in an automatic fashion.
+It is generally possible to hand-write solutions, so we explain some of the problems here, and the general approach to resolving them.
+
+### Tangent Types
+
+_**The Problem**_
+
+Suppose that you have a type such as:
+```julia
+mutable struct A
+    x::Float64
+    a::A
+    function A(x::Float64)
+        a = new(x)
+        a.a = a
+        return a
+    end
+end
+```
+
+This is a fairly canonical example of a self-referential type.
+There are a couple of things which will not work with it out-of-the-box.
+`tangent_type(A)` will produce a stack overflow error.
+To see this, note that it will in effect try to produce a tangent of type `Tangent{Tuple{tangent_type(A)}}` -- the circular dependency on the `tangent_type` function causes real problems here.
+
+_**The Solution**_
+
+In order to resolve this, you need to produce a tangent type by hand.
+You might go with something like
+```julia
+mutable struct TangentForA
+    x::Float64 # tangent type for Float64 is Float64
+    a::TangentForA
+    function TangentForA(x::Float64)
+        a = new(x)
+        a.a = a
+        return a
+    end
+end
+```
+The point here is that you can manually resolve the circular dependency using a data structure which mimics the primal type.
+You will, however, need to implement similar methods for `zero_tangent`, `randn_tangent`, etc, and presumably need to implement additional `getfield` and `setfield` rules which are specific to this type.
+
+### Circular References in General
+
+_**The Problem**_
+
+Consider a type of the form
+```julia
+mutable struct Foo
+    x
+    Foo() = new()
+end
+```
+In this instance, `tangent_type` will work fine because `Foo` does not directly reference itself in its definition.
+Moreover, general uses of `Foo` will be fine.
+
+However, it's possible to construct an instance of `Foo` with a circular reference:
+```julia
+f = Foo()
+f.x = f
+```
+This is actually fine provided we never attempt to call `zero_tangent` / `randn_tangent` / similar functionality on `f` once we've set its `x` field to itself.
+If we attempt to call such a function, we'll find ourselves with a stack overflow.
+
+_**The Solution**_
+This is a little tricker to handle.
+You could specialise `zero_tangent` etc for `Foo`, but this is something of a pain.
+Fortunately, it seems to be incredibly rare that this is ever a problem in practice.
+If we gain evidence that this _is_ often a problem in practice, we'll look into supporting `zero_tangent` etc automatically for this case.
+
+
+## Tangent Generation and Pointers
+
+```@meta
+DocTestSetup = quote
+    using Tapir
+end
+```
+
+_**The Problem**_
+
+
+In many use cases, a pointer provides the address of the start of a block of memory which has been allocated to e.g. store an array.
+However, we cannot get any of this context from the pointer itself -- by just looking at a pointer, I cannot know whether its purpose is to refer to the start of a large block of memory, some proportion of the way through a block of memory, or even to keep track of a single address.
+
+Recall that the tangent to a pointer is another pointer:
+```jldoctest
+julia> Tapir.tangent_type(Ptr{Float64})
+Ptr{Float64}
+```
+Plainly I cannot implement a method of `zero_tangent` for `Ptr{Float64}` because I don't know how much memory to allocate.
+
+This is, however, fine if a pointer appears half way through a function, having been derived from another data structure. e.g.
+```jldoctest
+function foo(x::Vector{Float64})
+    p = pointer(x, 2)
+    return unsafe_load(p)
+end
+
+rule = build_rrule(Tapir.TapirInterpreter(), Tuple{typeof(foo), Vector{Float64}})
+Tapir.value_and_gradient!!(rule, foo, [5.0, 4.0])
+
+# output
+(4.0, (NoTangent(), [0.0, 1.0]))
+```
+
+_**The Solution**_
+
+This is only really a problem for tangent / fdata / rdata generation functionality, such as `zero_tangent`.
+As a work-around, AD testing functionality permits users to pass in `CoDual`s.
+So if you are testing something involving a pointer, you will need to construct its tangent yourself, and pass a `CoDual` to e.g. `Tapir.TestUtils.test_derived_rule`.
+
+While pointers tend to be a low-level implementation detail in Julia code, you could in principle actually be interested in differentiating a function of a pointer.
+In this case, you will not be able to use `Tapir.value_and_gradient!!` as this requires the use of `zero_tangent`.
+Instead, you will need to use lower-level (internal) functionality, such as `Tapir.__value_and_gradient!!`, or use the rule interface directly.
+
+Honestly, your best bet is just to avoid differentiating functions whose arguments are pointers if you can.
+
+```@meta
+DocTestSetup = nothing
+```