API reference
DifferentiationInterfaceTest
— ModuleDifferentiationInterfaceTest
Testing and benchmarking utilities for automatic differentiation in Julia.
Exports
Entry points
DifferentiationInterfaceTest.test_differentiation
— Functiontest_differentiation(
+ backends::Vector{<:ADTypes.AbstractADType};
+ ...
+)
+test_differentiation(
+ backends::Vector{<:ADTypes.AbstractADType},
+ scenarios::Vector{<:AbstractScenario};
+ correctness,
+ type_stability,
+ call_count,
+ sparsity,
+ ref_backend,
+ detailed,
+ input_type,
+ output_type,
+ first_order,
+ second_order,
+ onearg,
+ twoarg,
+ inplace,
+ outofplace,
+ excluded,
+ logging,
+ isapprox,
+ atol,
+ rtol
+)
+
Cross-test a list of backends
on a list of scenarios
, running a variety of different tests.
Default arguments
scenarios::Vector{<:AbstractScenario}
: the output ofdefault_scenarios()
Keyword arguments
Testing:
correctness=true
: whether to compare the differentiation results with the theoretical values specified in each scenariotype_stability=false
: whether to check type stability with JET.jl (thanks to@test_opt
)sparsity
: whether to check sparsity of the jacobian / hessianref_backend
: if notnothing
, anADTypes.AbstractADType
object to use instead of the scenario-specific reference to provide true valuesdetailed=false
: whether to print a detailed or condensed test log
Filtering:
input_type=Any
,output_type=Any
: restrict scenario inputs / outputs to subtypes of thisfirst_order=true
,second_order=true
: include first order / second order operatorsonearg=true
,twoarg=true
: include one-argument / two-argument functionsinplace=true
,outofplace=true
: include in-place / out-of-place operators
Options:
logging=false
: whether to log progressisapprox=isapprox
: function used to compare objects, with the standard signatureisapprox(x, y; atol, rtol)
atol=0
: absolute precision for correctness testing (when comparing to the reference outputs)rtol=1e-3
: relative precision for correctness testing (when comparing to the reference outputs)
test_differentiation(
+ backend::ADTypes.AbstractADType,
+ args...;
+ kwargs...
+)
+
Shortcut for a single backend.
DifferentiationInterfaceTest.benchmark_differentiation
— Functionbenchmark_differentiation(
+ backends::Vector{<:ADTypes.AbstractADType};
+ ...
+) -> Vector{DifferentiationInterfaceTest.BenchmarkDataRow}
+benchmark_differentiation(
+ backends::Vector{<:ADTypes.AbstractADType},
+ scenarios::Vector{<:AbstractScenario};
+ input_type,
+ output_type,
+ first_order,
+ second_order,
+ onearg,
+ twoarg,
+ inplace,
+ outofplace,
+ excluded,
+ logging
+) -> Vector{DifferentiationInterfaceTest.BenchmarkDataRow}
+
Benchmark a list of backends
for a list of operators
on a list of scenarios
.
Keyword arguments
- filtering: same as
test_differentiation
for the filtering part. logging=false
: whether to log progress
Pre-made scenario lists
DifferentiationInterfaceTest.default_scenarios
— Functiondefault_scenarios()
Create a vector of AbstractScenario
s with standard array types.
DifferentiationInterfaceTest.sparse_scenarios
— Functionsparse_scenarios()
Create a vector of AbstractScenario
s with sparse array types, focused on sparse Jacobians and Hessians.
DifferentiationInterfaceTest.component_scenarios
— Functioncomponent_scenarios()
Create a vector of AbstractScenario
s with component array types from ComponentArrays.jl.
DifferentiationInterfaceTest.gpu_scenarios
— Functiongpu_scenarios()
Create a vector of AbstractScenario
s with GPU array types from JLArrays.jl.
DifferentiationInterfaceTest.static_scenarios
— Functionstatic_scenarios()
Create a vector of AbstractScenario
s with static array types from StaticArrays.jl.
Scenario types
DifferentiationInterfaceTest.AbstractScenario
— TypeAbstractScenario
Store a testing scenario composed of a function and its input + output.
This abstract type should never be used directly: construct one of the subtypes corresponding to the operator you want to test.
Subtypes
PushforwardScenario
PullbackScenario
DerivativeScenario
GradientScenario
JacobianScenario
SecondDerivativeScenario
HVPScenario
HessianScenario
Fields
All subtypes have the following fields:
f
: function to applyx
: primal inputy
: primal outputref
: reference to compare against
In addition, some subtypes contain an additional seed (dx
or dy
).
Constructor
If y
is provided, f
is interpreted as a 2-argument function f!(y, x) = nothing
. Otherwise, f
is interpreted as an 1-argument function f(x) = y
.
The reference keyword ref
should be a function that takes x
(and a potential seed dx
or dy
) to return the correct object.
The keyword place
should be either :inplace
or :outofplace
depending on what kind of operator must be tested.
DifferentiationInterfaceTest.PushforwardScenario
— TypePushforwardScenario(f; x, y, dx, ref, place)
See AbstractScenario
for details.
DifferentiationInterfaceTest.PullbackScenario
— TypePullbackScenario(f; x, y, dy, ref, place)
See AbstractScenario
for details.
DifferentiationInterfaceTest.DerivativeScenario
— TypeDerivativeScenario(f; x, y, ref, place)
See AbstractScenario
for details.
DifferentiationInterfaceTest.GradientScenario
— TypeGradientScenario(f; x, y, ref, place)
See AbstractScenario
for details.
DifferentiationInterfaceTest.JacobianScenario
— TypeJacobianScenario(f; x, y, ref, place)
See AbstractScenario
for details.
DifferentiationInterfaceTest.SecondDerivativeScenario
— TypeSecondDerivativeScenario(f; x, y, ref, place)
See AbstractScenario
for details.
DifferentiationInterfaceTest.HVPScenario
— TypeHVPScenario(f; x, y, dx, ref, place)
See AbstractScenario
for details.
DifferentiationInterfaceTest.HessianScenario
— TypeHessianScenario(f; x, y, ref, place)
See AbstractScenario
for details.
Internals
This is not part of the public API.
DifferentiationInterfaceTest.AutoZeroForward
— TypeAutoZeroForward <: ADTypes.AbstractADType
Trivial backend that sets all derivatives to zero. Used in testing and benchmarking.
DifferentiationInterfaceTest.AutoZeroReverse
— TypeAutoZeroReverse <: ADTypes.AbstractADType
Trivial backend that sets all derivatives to zero. Used in testing and benchmarking.
DifferentiationInterfaceTest.BenchmarkDataRow
— TypeBenchmarkDataRow
Ad-hoc storage type for differentiation benchmarking results. If you have a vector rows::Vector{BenchmarkDataRow}
, you can turn it into a DataFrame
as follows:
df = DataFrames.DataFrame(rows)
Fields
These are not part of the public API.
backend::String
mode::ADTypes.AbstractMode
scenario::Symbol
operator::Symbol
arguments::Int64
place::Symbol
func::Symbol
input_type::Type
output_type::Type
input_size::Tuple
output_size::Tuple
calls::Int64
samples::Int64
evals::Int64
time::Float64
bytes::Float64
allocs::Float64
compile_fraction::Float64
gc_fraction::Float64