Skip to content

Commit

Permalink
Format all markdown files with JuliaFormatter.jl (#169)
Browse files Browse the repository at this point in the history
* Format all markdown files with JuliaFormatter.jl

* Update docs/src/getting_started.md

* Update docs/src/dae.md

* Update docs/src/getting_started.md

* Update docs/src/getting_started.md

* Update docs/src/getting_started.md
  • Loading branch information
nathanaelbosch authored Aug 15, 2022
1 parent 8db3c25 commit 18fc101
Show file tree
Hide file tree
Showing 8 changed files with 170 additions and 100 deletions.
35 changes: 18 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,13 @@
# ProbNumDiffEq.jl



[![Stable](https://img.shields.io/badge/docs-stable-blue.svg)](https://nathanaelbosch.github.io/ProbNumDiffEq.jl/stable)
[![Dev](https://img.shields.io/badge/docs-dev-blue.svg)](https://nathanaelbosch.github.io/ProbNumDiffEq.jl/dev)
[![Build Status](https://github.com/nathanaelbosch/ProbNumDiffEq.jl/workflows/CI/badge.svg)](https://github.com/nathanaelbosch/ProbNumDiffEq.jl/actions)
[![Coverage](https://codecov.io/gh/nathanaelbosch/ProbNumDiffEq.jl/branch/main/graph/badge.svg?token=eufIemCGXn)](https://codecov.io/gh/nathanaelbosch/ProbNumDiffEq.jl)
[![Benchmarks](http://img.shields.io/badge/benchmarks-ipynb-blueviolet.svg)](./benchmarks/)

<!-- [![Code Style: Blue](https://img.shields.io/badge/code%20style-blue-4495d1.svg)](https://github.com/invenia/BlueStyle) -->
<!-- [![ColPrac: Contributor's Guide on Collaborative Practices for Community Packages](https://img.shields.io/badge/ColPrac-Contributor's%20Guide-blueviolet)](https://github.com/SciML/ColPrac) -->


![Banner](./examples/banner.svg?raw=true)

__ProbNumDiffEq.jl__ provides _probabilistic numerical_ ODE solvers to the
Expand All @@ -19,30 +16,32 @@ The implemented _ODE filters_ solve differential equations via Bayesian filterin

For a short intro video, check out our [poster presentation at JuliaCon2021](https://www.youtube.com/watch?v=EMFl6ytP3iQ).

---
* * *

__For more probabilistic numerics check out the [ProbNum](https://probnum.readthedocs.io/en/latest/) Python package.__
It implements probabilistic ODE solvers, but also probabilistic linear solvers, Bayesian quadrature, and many filtering and smoothing implementations.

---
* * *

## Installation

Run Julia, enter `]` to bring up Julia's package manager, and add the ProbNumDiffEq.jl package:

```
julia> ]
(v1.7) pkg> add ProbNumDiffEq.jl
```


## Example: Solving the FitzHugh-Nagumo ODE

```julia
using ProbNumDiffEq

# ODE definition as in DifferentialEquations.jl
function f(du, u, p, t)
a, b, c = p
du[1] = c*(u[1] - u[1]^3/3 + u[2])
du[2] = -(1/c)*(u[1] - a - b*u[2])
du[1] = c * (u[1] - u[1]^3 / 3 + u[2])
du[2] = -(1 / c) * (u[1] - a - b * u[2])
end
u0 = [-1.0, 1.0]
tspan = (0.0, 20.0)
Expand All @@ -56,20 +55,22 @@ sol = solve(prob, EK0(order=1), abstol=1e-2, reltol=1e-1)
using Plots
plot(sol, color=["#107D79" "#FF9933"])
```
![Fitzhugh-Nagumo Solution](./examples/fitzhughnagumo.svg?raw=true "Fitzhugh-Nagumo Solution")

![Fitzhugh-Nagumo Solution](./examples/fitzhughnagumo.svg?raw=true "Fitzhugh-Nagumo Solution")

## Benchmarks
- [Multi-Language Wrapper Benchmark](./benchmarks/multi-language-wrappers.ipynb):
ProbNumDiffEq.jl vs. OrdinaryDiffEq.jl, Hairer's FORTRAN solvers, Sundials, LSODA, MATLAB, and SciPy.

- [Multi-Language Wrapper Benchmark](./benchmarks/multi-language-wrappers.ipynb):
ProbNumDiffEq.jl vs. OrdinaryDiffEq.jl, Hairer's FORTRAN solvers, Sundials, LSODA, MATLAB, and SciPy.

## References

The main references _for this package_ include:
- M. Schober, S. Särkkä, and P. Hennig: **A Probabilistic Model for the Numerical Solution of Initial Value Problems** (2018) ([link](https://link.springer.com/article/10.1007/s11222-017-9798-7))
- F. Tronarp, H. Kersting, S. Särkkä, and P. Hennig: **Probabilistic Solutions To Ordinary Differential Equations As Non-Linear Bayesian Filtering: A New Perspective** (2019) ([link](https://link.springer.com/article/10.1007/s11222-019-09900-1))
- N. Krämer, P. Hennig: **Stable Implementation of Probabilistic ODE Solvers** (2020) ([link](https://arxiv.org/abs/2012.10106))
- N. Bosch, P. Hennig, F. Tronarp: **Calibrated Adaptive Probabilistic ODE Solvers** (2021) ([link](http://proceedings.mlr.press/v130/bosch21a.html))
- N. Bosch, F. Tronarp, P. Hennig: **Pick-and-Mix Information Operators for Probabilistic ODE Solvers** (2022) ([link](https://arxiv.org/abs/2110.10770))

- M. Schober, S. Särkkä, and P. Hennig: **A Probabilistic Model for the Numerical Solution of Initial Value Problems** (2018) ([link](https://link.springer.com/article/10.1007/s11222-017-9798-7))
- F. Tronarp, H. Kersting, S. Särkkä, and P. Hennig: **Probabilistic Solutions To Ordinary Differential Equations As Non-Linear Bayesian Filtering: A New Perspective** (2019) ([link](https://link.springer.com/article/10.1007/s11222-019-09900-1))
- N. Krämer, P. Hennig: **Stable Implementation of Probabilistic ODE Solvers** (2020) ([link](https://arxiv.org/abs/2012.10106))
- N. Bosch, P. Hennig, F. Tronarp: **Calibrated Adaptive Probabilistic ODE Solvers** (2021) ([link](http://proceedings.mlr.press/v130/bosch21a.html))
- N. Bosch, F. Tronarp, P. Hennig: **Pick-and-Mix Information Operators for Probabilistic ODE Solvers** (2022) ([link](https://arxiv.org/abs/2110.10770))

A more extensive list of references relevant to ODE filters is provided [here](https://nathanaelbosch.github.io/ProbNumDiffEq.jl/stable/#References).
41 changes: 26 additions & 15 deletions docs/src/dae.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,46 @@
# Solving DAEs with Probabilistic Numerics

ProbNumDiffEq.jl provides probabilistic numerical solvers for _differential algebraic equations_ (DAEs).
Currently, we recommend using the semi-implicit `EK1` algorithm.

!!! note

For a more general tutorial on DAEs, solved with classic solvers, check out the
[DifferentialEquations.jl DAE tutorial](https://diffeq.sciml.ai/stable/tutorials/dae_example/).


#### Solving a Mass-Matrix DAE with the `EK1`

```@example 2
using ProbNumDiffEq, Plots
function rober(du,u,p,t)
y₁,y₂,y₃ = u
k₁,k₂,k₃ = p
du[1] = -k₁*y₁ + k₃*y₂*y₃
du[2] = k₁*y₁ - k₃*y₂*y₃ - k₂*y₂^2
du[3] = y₁ + y₂ + y₃ - 1
nothing
function rober(du, u, p, t)
y₁, y₂, y₃ = u
k₁, k₂, k₃ = p
du[1] = -k₁ * y₁ + k₃ * y₂ * y₃
du[2] = k₁ * y₁ - k₃ * y₂ * y₃ - k₂ * y₂^2
du[3] = y₁ + y₂ + y₃ - 1
nothing
end
M = [1. 0 0
0 1. 0
0 0 0]
f = ODEFunction(rober,mass_matrix=M)
M = [1.0 0 0
0 1.0 0
0 0 0]
f = ODEFunction(rober, mass_matrix=M)
prob_mm = ODEProblem(f, [1.0, 0.0, 0.0], (0.0, 1e5), (0.04, 3e7, 1e4))
using Logging; Logging.disable_logging(Logging.Warn) # hide
using Logging; Logging.disable_logging(Logging.Warn); # hide
sol = solve(prob_mm, EK1(), reltol=1e-8, abstol=1e-8)
Logging.disable_logging(Logging.Debug) # hide
plot(sol, xscale=:log10, tspan=(1e-6, 1e5), layout=(3,1), legend=false, ylabel=["u₁(t)" "u₂(t)" "u₃(t)"], xlabel=["" "" "t"], denseplot=false)
plot(
sol,
xscale=:log10,
tspan=(1e-6, 1e5),
layout=(3, 1),
legend=false,
ylabel=["u₁(t)" "u₂(t)" "u₃(t)"],
xlabel=["" "" "t"],
denseplot=false,
)
```


### References

[1] N. Bosch, F. Tronarp, P. Hennig: **Pick-and-Mix Information Operators for Probabilistic ODE Solvers** (2022) ([link](https://arxiv.org/abs/2110.10770))
54 changes: 36 additions & 18 deletions docs/src/dynamical_odes.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,39 @@
# Second Order ODEs and Energy Preservation
In this tutorial we consider an _energy-preserving_, physical dynamical system, given by a _second-order_ ODE.

In this tutorial we consider an _energy-preserving_, physical dynamical system, given by a _second-order_ ODE.

#### TL;DR:
1. To _efficiently_ solve second-order ODEs, just define the problem as a `SecondOrderODEProblem`.
2. To preserve constant quantities, use the `ManifoldUpdate` callback; same syntax as
[DiffEqCallback.jl's `ManifoldProjection`](https://diffeq.sciml.ai/stable/features/callback_library/#Manifold-Conservation-and-Projection).

1. To _efficiently_ solve second-order ODEs, just define the problem as a `SecondOrderODEProblem`.
2. To preserve constant quantities, use the `ManifoldUpdate` callback; same syntax as
[DiffEqCallback.jl's `ManifoldProjection`](https://diffeq.sciml.ai/stable/features/callback_library/#Manifold-Conservation-and-Projection).

## Simulating the Hénon-Heiles system

The Hénon-Heiles model describes the motion of a star around a galactic center, restricted to a plane.
It is given by a second-order ODE

```math
\begin{aligned}
\ddot{x} &= - x - 2 x y \\
\ddot{y} &= y^2 - y - x^2.
\end{aligned}
```

Our goal is to numerically simulate this system on a time span
``t \in [0, T]``, starting with initial values
``x(0)=0``, ``y(0) = 0.1``, ``\dot{x}(0) = 0.5``, ``\dot{y}(0) = 0``.


### Transforming the problem into a first-order ODE

A very common approach is to first transform the problem into a first-order ODE by introducing a new variable

```math
u = [dx,dy,x,y],
```

to obtain

```math
\begin{aligned}
\dot{u}_1(t) &= - u_3 - 2 u_3 u_4 \\
Expand All @@ -42,42 +48,46 @@ This first-order ODE can then be solved using any conventional ODE solver - incl
```@example 2
using ProbNumDiffEq, Plots
function Hénon_Heiles(du,u,p,t)
du[1] = -u[3] - 2*u[3]*u[4]
du[2] = u[4]^2 - u[4] -u[3]^2
function Hénon_Heiles(du, u, p, t)
du[1] = -u[3] - 2 * u[3] * u[4]
du[2] = u[4]^2 - u[4] - u[3]^2
du[3] = u[1]
du[4] = u[2]
end
u0, du0 = [0.0, 0.1], [0.5, 0.0]
tspan = (0.0, 100.0)
prob = ODEProblem(Hénon_Heiles, [du0; u0], tspan)
sol = solve(prob, EK1());
plot(sol, vars=(3,4)) # where `vars=(3,4)` is used to plot x agains y
plot(sol, vars=(3, 4)) # where `vars=(3,4)` is used to plot x agains y
```

### Solving the second-order ODE directly

Instead of first transforming the problem, we can also solve it directly as a second-order ODE, by defining it as a `SecondOrderODEProblem`.

!!! note

The `SecondOrderODEProblem` type is not defined in ProbNumDiffEq.jl but is provided by SciMLBase.jl.
For more information, check out the DifferentialEquations.jl documentation on [Dynamical, Hamiltonian and 2nd Order ODE Problems](https://diffeq.sciml.ai/stable/types/dynamical_types/).

```@example 2
function Hénon_Heiles2(ddu,du,u,p,t)
ddu[1] = -u[1] - 2*u[1]*u[2]
ddu[2] = u[2]^2 - u[2] -u[1]^2
function Hénon_Heiles2(ddu, du, u, p, t)
ddu[1] = -u[1] - 2 * u[1] * u[2]
ddu[2] = u[2]^2 - u[2] - u[1]^2
end
prob2 = SecondOrderODEProblem(Hénon_Heiles2, du0, u0, tspan)
sol2 = solve(prob2, EK1());
plot(sol2, vars=(3,4))
plot(sol2, vars=(3, 4))
```

### Benchmark: Solving second order ODEs is _faster_

Solving second-order ODEs is not just a matter of convenience - in fact, SciMLBase's `SecondOrderODEProblem` is neatly designed in such a way that all the classic solvers from OrdinaryDiffEq.jl can handle it by solving the corresponding first-order ODE.
But, transforming the ODE to first order increases the dimensionality of the problem, and comes therefore at increased computational cost; this also motivates [classic specialized solvers for second-order ODEs](https://diffeq.sciml.ai/stable/solvers/dynamical_solve/).

The probabilistic numerical solvers from ProbNumDiffEq.jl have the same internal state representation for first and second order ODEs; all that changes is the _measurement model_ [1].
As a result, we can use the `EK1` both for first and second order ODEs, but it automatically specializes on the latter to provide a __2x performance boost__:

```
julia> @btime solve(prob, EK1(order=3), adaptive=false, dt=1e-2);
766.312 ms (400362 allocations: 173.38 MiB)
Expand All @@ -86,10 +96,11 @@ julia> @btime solve(prob2, EK1(order=4), adaptive=false, dt=1e-2);
388.301 ms (510676 allocations: 102.78 MiB)
```


## Energy preservation

In addition to the ODE given above, we know that the solution of the Hénon-Heiles model has to _preserve energy_ over time.
The total energy can be expressed as the sum of the potential and kinetic energies, given by

```math
\begin{aligned}
\operatorname{PotentialEnergy}(x,y) &= \frac{1}{2} \left( x^2 + y^2 + 2 x^2 y - \frac{2y^3}{3} \right), \\
Expand All @@ -98,37 +109,44 @@ The total energy can be expressed as the sum of the potential and kinetic energi
```

In code:

```@example 2
PotentialEnergy(x,y) = 1//2 * (x^2 + y^2 + 2x^2*y - 2//3 * y^3)
KineticEnergy(dx,dy) = 1//2 * (dx^2 + dy^2)
E(dx,dy,x,y) = PotentialEnergy(x,y) + KineticEnergy(dx,dy)
PotentialEnergy(x, y) = 1 // 2 * (x^2 + y^2 + 2x^2 * y - 2 // 3 * y^3)
KineticEnergy(dx, dy) = 1 // 2 * (dx^2 + dy^2)
E(dx, dy, x, y) = PotentialEnergy(x, y) + KineticEnergy(dx, dy)
E(u) = E(u...); # convenient shorthand
```

So, let's have a look at how the total energy changes over time when we numerically simulate the Hénon-Heiles model over a long period of time:
Standard solve

```@example 2
longprob = remake(prob2, tspan=(0.0, 1e3))
longsol = solve(longprob, EK1(smooth=false), dense=false)
plot(longsol.t, E.(longsol.u))
```

It visibly loses energy over time, from an initial 0.12967 to a final 0.12899.
Let's fix this to get a physically more meaningful solution.

### Energy preservation with the `ManifoldUpdate` callback

In the language of ODE filters, preserving energy over time amounts to just another measurement model [1].
The most convenient way of updating on this additional zero measurement with ProbNumDiffEq.jl is with the `ManifoldUpdate` callback.

!!! note

The `ManifoldUpdate` callback can be thought of a probabilistic counterpart to the [`ManifoldProjection`](https://diffeq.sciml.ai/stable/features/callback_library/#Manifold-Conservation-and-Projection) callback provided by DiffEqCallbacks.jl.

To do so, first define a (vector-valued) residual function, here chosen to be the difference between the current energy and the initial energy, and build a `ManifoldUpdate` callback

```@example 2
residual(u) = [E(u) - E(du0..., u0...)]
cb = ManifoldUpdate(residual)
```

Then, solve the ODE with this callback

```@example 2
longsol_preserving = solve(longprob, EK1(smooth=false), dense=false, callback=cb)
plot(longsol.t, E.(longsol.u))
Expand All @@ -137,6 +155,6 @@ plot!(longsol_preserving.t, E.(longsol_preserving.u))

Voilà! With the `ManifoldUpdate` callback we could preserve the energy over time and obtain a more truthful probabilistic numerical long-term simulation of the Hénon-Heiles model.


#### References

[1] N. Bosch, F. Tronarp, P. Hennig: **Pick-and-Mix Information Operators for Probabilistic ODE Solvers** (2022)
4 changes: 4 additions & 0 deletions docs/src/filtering.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
# Gaussian Filtering and Smoothing

## Predict

```@docs
ProbNumDiffEq.predict
ProbNumDiffEq.predict!
```

## Update

```@docs
ProbNumDiffEq.update
ProbNumDiffEq.update!
```

## Smooth

```@docs
ProbNumDiffEq.smooth
ProbNumDiffEq.smooth!
Expand Down
Loading

0 comments on commit 18fc101

Please sign in to comment.