From 0f48798924c03120e1621c88026b1ee45b971fec Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Mon, 29 Apr 2024 05:43:25 +0000 Subject: [PATCH] build based on 2fe6c41 --- .../dev/.documenter-siteinfo.json | 2 +- .../dev/api/index.html | 8 ++++---- DifferentiationInterfaceTest/dev/index.html | 2 +- .../dev/tutorial/index.html | 20 +++++++++---------- 4 files changed, 16 insertions(+), 16 deletions(-) diff --git a/DifferentiationInterfaceTest/dev/.documenter-siteinfo.json b/DifferentiationInterfaceTest/dev/.documenter-siteinfo.json index 7e30dea08..783304a20 100644 --- a/DifferentiationInterfaceTest/dev/.documenter-siteinfo.json +++ b/DifferentiationInterfaceTest/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.2","generation_timestamp":"2024-04-28T16:35:30","documenter_version":"1.4.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.2","generation_timestamp":"2024-04-29T05:43:21","documenter_version":"1.4.0"}} \ No newline at end of file diff --git a/DifferentiationInterfaceTest/dev/api/index.html b/DifferentiationInterfaceTest/dev/api/index.html index d8bb30c79..5c4640ba0 100644 --- a/DifferentiationInterfaceTest/dev/api/index.html +++ b/DifferentiationInterfaceTest/dev/api/index.html @@ -1,5 +1,5 @@ -API reference · DifferentiationInterfaceTest.jl
GitHub

API reference

Entry points

DifferentiationInterfaceTest.test_differentiation — Function
test_differentiation(
+API reference · DifferentiationInterfaceTest.jl
GitHub
API reference
Entry points
DifferentiationInterfaceTest.test_differentiation — Function
test_differentiation(
     backends::Vector{<:ADTypes.AbstractADType};
     ...
 )
@@ -22,12 +22,12 @@
     atol,
     rtol
 )
-
Cross-test a list of backends on a list of scenarios, running a variety of different tests.
Default arguments
Keyword arguments
Testing:
  • correctness=true: whether to compare the differentiation results with the theoretical values specified in each scenario
  • type_stability=false: whether to check type stability with JET.jl (thanks to @test_opt)
  • sparsity: whether to check sparsity of the jacobian / hessian
  • ref_backend: if not nothing, an ADTypes.AbstractADType object to use instead of the scenario-specific reference to provide true values
  • detailed=false: whether to print a detailed or condensed test log
Filtering:
  • input_type=Any: restrict scenario inputs to subtypes of this
  • output_type=Any: restrict scenario outputs to subtypes of this
  • first_order=true: include first order operators
  • second_order=true: include second order operators
Options:
  • logging=false: whether to log progress
  • isapprox=isapprox: function used to compare objects, with the standard signature isapprox(x, y; atol, rtol)
  • atol=0: absolute precision for correctness testing (when comparing to the reference outputs)
  • rtol=1e-3: relative precision for correctness testing (when comparing to the reference outputs)
source
test_differentiation(
+
Cross-test a list of backends on a list of scenarios, running a variety of different tests.
Default arguments
Keyword arguments
Testing:
  • correctness=true: whether to compare the differentiation results with the theoretical values specified in each scenario
  • type_stability=false: whether to check type stability with JET.jl (thanks to @test_opt)
  • sparsity: whether to check sparsity of the jacobian / hessian
  • ref_backend: if not nothing, an ADTypes.AbstractADType object to use instead of the scenario-specific reference to provide true values
  • detailed=false: whether to print a detailed or condensed test log
Filtering:
  • input_type=Any: restrict scenario inputs to subtypes of this
  • output_type=Any: restrict scenario outputs to subtypes of this
  • first_order=true: include first order operators
  • second_order=true: include second order operators
Options:
  • logging=false: whether to log progress
  • isapprox=isapprox: function used to compare objects, with the standard signature isapprox(x, y; atol, rtol)
  • atol=0: absolute precision for correctness testing (when comparing to the reference outputs)
  • rtol=1e-3: relative precision for correctness testing (when comparing to the reference outputs)
source
test_differentiation(
     backend::ADTypes.AbstractADType,
     args...;
     kwargs...
 )
-
Shortcut for a single backend.
source
DifferentiationInterfaceTest.benchmark_differentiation — Function
benchmark_differentiation(
     backends::Vector{<:ADTypes.AbstractADType};
     ...
 ) -> Vector{DifferentiationInterfaceTest.BenchmarkDataRow}
@@ -41,4 +41,4 @@
     excluded,
     logging
 ) -> Vector{DifferentiationInterfaceTest.BenchmarkDataRow}
-
Benchmark a list of backends for a list of operators on a list of scenarios.
Keyword arguments
  • filtering: same as test_differentiation for the filtering part.
  • logging=false: whether to log progress
source
Pre-made scenario lists
Scenario types
DifferentiationInterfaceTest.AbstractScenario — Type
AbstractScenario
Store a testing scenario composed of a function and its input + output.
This abstract type should never be used directly: construct one of the subtypes corresponding to the operator you want to test.
Subtypes
Fields
All subtypes have the following fields:
  • f: function to apply
  • x: primal input
  • y: primal output
  • ref: reference to compare against
In addition, some subtypes contain an additional seed (dx or dy).
Constructor
If y is provided, f is interpreted as a 2-argument function f!(y, x) = nothing. Otherwise, f is interpreted as an 1-argument function f(x) = y.
The reference keyword ref should be a function that takes x (and a potential seed dx or dy) to return the correct object.
The operator behavior keyword operator should be either :inplace or :outofplace depending on what must be tested.
source
Internals
This is not part of the public API.
DifferentiationInterfaceTest.BenchmarkDataRow — Type
BenchmarkDataRow
Ad-hoc storage type for differentiation benchmarking results. If you have a vector rows::Vector{BenchmarkDataRow}, you can turn it into a DataFrame as follows:
df = DataFrames.DataFrame(rows)
Fields
These are not part of the public API.
  • backend::String
  • mode::ADTypes.AbstractMode
  • scenario::Symbol
  • operator::Symbol
  • func::Symbol
  • arguments::Int64
  • input_type::Type
  • output_type::Type
  • input_size::Tuple
  • output_size::Tuple
  • calls::Int64
  • samples::Int64
  • evals::Int64
  • time::Float64
  • bytes::Float64
  • allocs::Float64
  • compile_fraction::Float64
  • gc_fraction::Float64
source

+

Benchmark a list of backends for a list of operators on a list of scenarios.

Keyword arguments

  • filtering: same as test_differentiation for the filtering part.
  • logging=false: whether to log progress
source

Pre-made scenario lists

Scenario types

DifferentiationInterfaceTest.AbstractScenario — Type
AbstractScenario

Store a testing scenario composed of a function and its input + output.

This abstract type should never be used directly: construct one of the subtypes corresponding to the operator you want to test.

Subtypes

Fields

All subtypes have the following fields:

  • f: function to apply
  • x: primal input
  • y: primal output
  • ref: reference to compare against

In addition, some subtypes contain an additional seed (dx or dy).

Constructor

If y is provided, f is interpreted as a 2-argument function f!(y, x) = nothing. Otherwise, f is interpreted as an 1-argument function f(x) = y.

The reference keyword ref should be a function that takes x (and a potential seed dx or dy) to return the correct object.

The operator behavior keyword operator should be either :inplace or :outofplace depending on what must be tested.

source

Internals

This is not part of the public API.

DifferentiationInterfaceTest.BenchmarkDataRow — Type
BenchmarkDataRow

Ad-hoc storage type for differentiation benchmarking results. If you have a vector rows::Vector{BenchmarkDataRow}, you can turn it into a DataFrame as follows:

df = DataFrames.DataFrame(rows)

Fields

These are not part of the public API.

  • backend::String

  • mode::ADTypes.AbstractMode

  • scenario::Symbol

  • operator::Symbol

  • func::Symbol

  • arguments::Int64

  • input_type::Type

  • output_type::Type

  • input_size::Tuple

  • output_size::Tuple

  • calls::Int64

  • samples::Int64

  • evals::Int64

  • time::Float64

  • bytes::Float64

  • allocs::Float64

  • compile_fraction::Float64

  • gc_fraction::Float64

source
diff --git a/DifferentiationInterfaceTest/dev/index.html b/DifferentiationInterfaceTest/dev/index.html index f4464a659..64d3030f6 100644 --- a/DifferentiationInterfaceTest/dev/index.html +++ b/DifferentiationInterfaceTest/dev/index.html @@ -11,4 +11,4 @@ julia> Pkg.add( url="https://github.com/gdalle/DifferentiationInterface.jl", subdir="DifferentiationInterfaceTest" - ) + ) diff --git a/DifferentiationInterfaceTest/dev/tutorial/index.html b/DifferentiationInterfaceTest/dev/tutorial/index.html index 9637a1deb..6b609ba15 100644 --- a/DifferentiationInterfaceTest/dev/tutorial/index.html +++ b/DifferentiationInterfaceTest/dev/tutorial/index.html @@ -11,15 +11,15 @@ type_stability=false, # checks type stability with JET.jl detailed=true, # prints a detailed test set )Test Summary: | Pass Total Time -Testing correctness | 32 32 37.4s - ForwardDiff (forward) | 16 16 3.9s - GradientScenario | 16 16 3.8s - GradientScenario{1,inplace} f : Vector{Float32} -> Float32 | 8 8 2.0s +Testing correctness | 32 32 35.3s + ForwardDiff (forward) | 16 16 3.6s + GradientScenario | 16 16 3.6s + GradientScenario{1,inplace} f : Vector{Float32} -> Float32 | 8 8 1.8s GradientScenario{1,inplace} f : Matrix{Float64} -> Float64 | 8 8 1.8s - Enzyme (reverse) | 16 16 33.5s - GradientScenario | 16 16 33.5s - GradientScenario{1,inplace} f : Vector{Float32} -> Float32 | 8 8 32.0s - GradientScenario{1,inplace} f : Matrix{Float64} -> Float64 | 8 8 1.6s

If you are too lazy to manually specify the reference, you can also provide an AD backend as the ref_backend keyword argument, which will serve as the ground truth for comparison.

Benchmarking

Once you are confident that your backends give the correct answers, you probably want to compare their performance. This is made easy by the benchmark_differentiation function, whose syntax should feel familiar:

julia> benchmark_result = benchmark_differentiation(backends, scenarios);

The resulting object is a Vector of structs, which can easily be converted into a DataFrame from DataFrames.jl:

julia> df = DataFrames.DataFrame(benchmark_result)12×18 DataFrame
+  Enzyme (reverse)                                               |   16     16  31.7s
+    GradientScenario                                             |   16     16  31.7s
+      GradientScenario{1,inplace} f : Vector{Float32} -> Float32 |    8      8  30.2s
+      GradientScenario{1,inplace} f : Matrix{Float64} -> Float64 |    8      8   1.5s

If you are too lazy to manually specify the reference, you can also provide an AD backend as the ref_backend keyword argument, which will serve as the ground truth for comparison.

Benchmarking

Once you are confident that your backends give the correct answers, you probably want to compare their performance. This is made easy by the benchmark_differentiation function, whose syntax should feel familiar:

julia> benchmark_result = benchmark_differentiation(backends, scenarios);

The resulting object is a Vector of structs, which can easily be converted into a DataFrame from DataFrames.jl:

julia> df = DataFrames.DataFrame(benchmark_result)12×18 DataFrame
  Row │ backend                mode           scenario          operator        ⋯
      │ String                 Abstract…      Symbol            Symbol          ⋯
 ─────┼──────────────────────────────────────────────────────────────────────────
@@ -53,5 +53,5 @@
     formatters=formatter
 )
 
-Markdown.parse(table)
backendmodescenariooperatorfuncargumentsinput_typeoutput_typeinput_sizeoutput_sizecallssamplesevalstimebytesallocscompile_fractiongc_fraction
ForwardDiff (forward)ForwardMode()GradientScenarioprepare_gradientf1Vector{Float32}Float32(3,)()0111.2e-055.3e+0211.00.00.0
ForwardDiff (forward)ForwardMode()GradientScenariovalue_and_gradient!f1Vector{Float32}Float32(3,)()129641711.5e-071.3e+023.00.00.0
ForwardDiff (forward)ForwardMode()GradientScenariogradient!f1Vector{Float32}Float32(3,)()125632121.3e-076.5e+011.00.00.0
ForwardDiff (forward)ForwardMode()GradientScenarioprepare_gradientf1Matrix{Float64}Float64(3, 2)()0111.1e-051.8e+0311.00.00.0
ForwardDiff (forward)ForwardMode()GradientScenariovalue_and_gradient!f1Matrix{Float64}Float64(3, 2)()12611972.8e-075.3e+027.10.00.0
ForwardDiff (forward)ForwardMode()GradientScenariogradient!f1Matrix{Float64}Float64(3, 2)()12573982.8e-074.7e+025.10.00.0
Enzyme (reverse)ReverseMode()GradientScenarioprepare_gradientf1Vector{Float32}Float32(3,)()0112.2e-070.0e+000.00.00.0
Enzyme (reverse)ReverseMode()GradientScenariovalue_and_gradient!f1Vector{Float32}Float32(3,)()1270246.5e-061.8e+0339.03.7051e-50.0
Enzyme (reverse)ReverseMode()GradientScenariogradient!f1Vector{Float32}Float32(3,)()1296074.1e-061.6e+0329.00.005120760.0
Enzyme (reverse)ReverseMode()GradientScenarioprepare_gradientf1Matrix{Float64}Float64(3, 2)()0111.4e-070.0e+000.00.00.0
Enzyme (reverse)ReverseMode()GradientScenariovalue_and_gradient!f1Matrix{Float64}Float64(3, 2)()1378428.8e-061.9e+0341.04.34637e-50.0
Enzyme (reverse)ReverseMode()GradientScenariogradient!f1Matrix{Float64}Float64(3, 2)()1356154.7e-061.6e+0329.40.0041840.0
-
+Markdown.parse(table)
backendmodescenariooperatorfuncargumentsinput_typeoutput_typeinput_sizeoutput_sizecallssamplesevalstimebytesallocscompile_fractiongc_fraction
ForwardDiff (forward)ForwardMode()GradientScenarioprepare_gradientf1Vector{Float32}Float32(3,)()0116.0e-065.3e+0211.00.00.0
ForwardDiff (forward)ForwardMode()GradientScenariovalue_and_gradient!f1Vector{Float32}Float32(3,)()122731991.5e-071.3e+023.00.00.0
ForwardDiff (forward)ForwardMode()GradientScenariogradient!f1Vector{Float32}Float32(3,)()128942201.3e-076.5e+011.00.00.0
ForwardDiff (forward)ForwardMode()GradientScenarioprepare_gradientf1Matrix{Float64}Float64(3, 2)()0117.4e-061.8e+0311.00.00.0
ForwardDiff (forward)ForwardMode()GradientScenariovalue_and_gradient!f1Matrix{Float64}Float64(3, 2)()12738902.8e-075.3e+027.10.00.0
ForwardDiff (forward)ForwardMode()GradientScenariogradient!f1Matrix{Float64}Float64(3, 2)()17726262.8e-074.7e+025.30.00.0
Enzyme (reverse)ReverseMode()GradientScenarioprepare_gradientf1Vector{Float32}Float32(3,)()0111.4e-070.0e+000.00.00.0
Enzyme (reverse)ReverseMode()GradientScenariovalue_and_gradient!f1Vector{Float32}Float32(3,)()1266755.6e-061.8e+0338.62.59713e-50.0
Enzyme (reverse)ReverseMode()GradientScenariogradient!f1Vector{Float32}Float32(3,)()1323464.4e-061.6e+0329.20.004836260.0
Enzyme (reverse)ReverseMode()GradientScenarioprepare_gradientf1Matrix{Float64}Float64(3, 2)()0111.5e-070.0e+000.00.00.0
Enzyme (reverse)ReverseMode()GradientScenariovalue_and_gradient!f1Matrix{Float64}Float64(3, 2)()1299955.0e-061.8e+0338.64.95133e-50.0
Enzyme (reverse)ReverseMode()GradientScenariogradient!f1Matrix{Float64}Float64(3, 2)()1324374.0e-061.6e+0329.00.004691410.0
+