Add developers doc; accordingly rename abstract supertypes (#72)

* start devdocs * change supertype docstring for changes * rename IndicatorsChangesConfig -> ChangesConfig * rename IndicatorsChangesResults -> ChangesResults * finish core steps for new change metric * rename estimate_indicator_changes -> estimate_changes * finish the devdocs * rename TransitionsSIgnificance -> SignificanceConfig * add devdocs to the doc build * rename SignificanceConfig -> Significance; not confuse with ChangeConfig
JuliaDynamics · Feb 19, 2024 · b4fc84c · b4fc84c
1 parent abde970
commit b4fc84c
Show file tree

Hide file tree

Showing 20 changed files with 134 additions and 77 deletions.
diff --git a/benchmark/benchmarks.jl b/benchmark/benchmarks.jl
@@ -56,11 +56,11 @@ config = SlidingWindowConfig(indicators, change_metrics;
     width_cha = 100, stride_cha = 1, whichtime = last,
 )
 
-res = estimate_indicator_changes(config, x, t)
+res = estimate_changes(config, x, t)
 signif = SurrogatesSignificance(n = 100, tail = :both, p = 0.1)
 flags = significant_transitions(res, signif)
 
-@btime estimate_indicator_changes($config, $x, $t)
+@btime estimate_changes($config, $x, $t)
 # 141.557 μs (4 allocations: 40.56 KiB)
 @btime significant_transitions($res, $signif)
 #=

diff --git a/benchmark/swapped_loops.jl b/benchmark/swapped_loops.jl
@@ -6,7 +6,7 @@ x = rand(n)
 indicators = (var, ar1_whitenoise)
 change_metrics = RidgeRegressionSlope()
 config = SlidingWindowConfig(indicators, change_metrics, stride_ind = 2, stride_cha = 10)
-results = estimate_indicator_changes(config, x, t)
+results = estimate_changes(config, x, t)
 signif = SurrogatesSignificance()
 @btime significant_transitions($results, $signif)
 #=
@@ -24,7 +24,7 @@ tseg_start = 100 .+ range(0, 0.9 * n, length = nseg)
 tseg_end = 900 .+ range(0, 0.9 * n, length = nseg)
 config = SegmentedWindowConfig(indicators, change_metrics, collect(tseg_start),
     collect(tseg_end), min_width_cha = 50)
-results = estimate_indicator_changes(config, x, t)
+results = estimate_changes(config, x, t)
 flags = significant_transitions(results, signif)
 @btime significant_transitions($results, $signif)
 #=

diff --git a/docs/make.jl b/docs/make.jl
@@ -19,6 +19,7 @@ pages = [
     "Examples" => example_pages,
     "api.md",
     "refs.md",
+    "devdocs.md",
 ]
 
 import Downloads

diff --git a/docs/src/api.md b/docs/src/api.md
@@ -3,11 +3,11 @@
 ## Main analysis functions
 
 ```@docs
-IndicatorsChangesConfig
+ChangesConfig
 SlidingWindowConfig
 SegmentedWindowConfig
-estimate_indicator_changes
-IndicatorsChangesResults
+estimate_changes
+ChangesResults
 SlidingWindowResults
 SegmentedWindowResults
 ```
@@ -16,7 +16,7 @@ SegmentedWindowResults
 
 ```@docs
 significant_transitions
-TransitionsSignificance
+Significance
 SurrogatesSignificance
 ThresholdSignificance
 SigmaSignificance

diff --git a/docs/src/devdocs.md b/docs/src/devdocs.md
@@ -0,0 +1,50 @@
+# Developer's documentation
+
+This documentation addresses users that would like to contribute to the software by either solving bugs, improving documentation, or adding new methods.
+All contributions come in the form of Pull Requests, for which we strongly advise to follow [good practices in scientific code](https://github.com/JuliaDynamics/GoodScientificCodeWorkshop), which means that they properly formatted, documented, tested, etc.
+
+## New indicators or change metrics
+
+As explained already in e.g., [`SlidingIndicatorConfig`](@ref), new indicators or change metrics are standard Julia functions, so you only need to define such a function (and document it, test it, etc.).
+
+## New pipeline for estimating changes
+
+This means to contribute a fundamentally new "pipeline" for estimating/detecting
+transitions in timeseries. This new pipeline defines what a "transition" means.
+To add a new pipeline follow these steps:
+
+1. Define a new subtype of [`ChangesConfig`](@ref)
+2. Define a new subtype of [`ChangesResults`](@ref)
+3. Add a method for [`estimate_changes`](@ref) which accepts
+   the new `ChangesConfig` subtype you defined and
+   returns the `ChangesResults` subtype you defined.
+
+And that's it!
+
+_Optionally_ you can further extend:
+
+- `TransitionsInTimeseries.plot_indicator_changes` with the new `ChangesResults` type you defined. Note the plotting functions are in the `ext` folder of the repository.
+- `significant_transitions(res::ChangesResults, signif::Significance)`
+  with your new `ChangesResults` type and as many `Significance`
+  subtypes you have the capacity to extend for.
+
+
+## New pipeline for estimating significance
+
+Statistically significant changes are "transitions".
+However, what "significant" means is not universal. There are different ways to
+test for significance and TransitionsInTimeseries.jl allows various methods.
+
+To add a new pipeline for estimating significance follow these steps:
+
+1. Define a new subtype of [`Significance`](@ref)
+2. Extend `significant_transitions(res::ChangesResults, signif::Significance)`
+   with your new type and as many `ChangesResults` subtypes as you have
+   capacity for.
+
+And that's it! Unfortunately so far we have found no way to make the
+`significant_transitions` code agnostic of the changes result, so you would
+need to add a method manually for every `ChangesResults`.
+
+_Optionally_ you can further extend `TransitionsInTimeseries.plot_significance!`
+(which you can find in the `ext` folder) for visualizing the significance results.
diff --git a/docs/src/examples/do-events.jl b/docs/src/examples/do-events.jl
@@ -184,7 +184,7 @@ tseg_end = t_rasmussen[2:end] .- 200
 config = SegmentedWindowConfig(indicators, change_metrics,
     tseg_start, tseg_end; whichtime = last, width_ind = Int(200÷dt),
     min_width_cha = 100)        # require >=100 data points to estimate change metric
-results = estimate_indicator_changes(config, r, t)
+results = estimate_changes(config, r, t)
 signif = SurrogatesSignificance(n = 1000, tail = [:right, :right], rng = Xoshiro(1995))
 flags = significant_transitions(results, signif)
 
@@ -246,7 +246,7 @@ tseg_end = t_rasmussen[2:end] .- 700    # stop analysis 500 years earlier than b
 config = SegmentedWindowConfig(indicators, change_metrics,
     tseg_start, tseg_end, whichtime = last, width_ind = Int(200÷dt),
     min_width_cha = 100)
-results = estimate_indicator_changes(config, r, t)
+results = estimate_changes(config, r, t)
 signif = SurrogatesSignificance(n = 1000, tail = [:right, :right], rng = Xoshiro(1995))
 flags = significant_transitions(results, signif)
 fig, axs = plot_do(t3, x3, tloess, xloess, t, r, t_rasmussen, xlims, xticks)
@@ -297,7 +297,7 @@ for j in 1:2
     config = SegmentedWindowConfig(
         indicators, change_metrics, tseg_start[j], tseg_end[j],
         whichtime = last, width_ind = Int(200÷dt), min_width_cha = 100)
-    results = estimate_indicator_changes(config, r, t)
+    results = estimate_changes(config, r, t)
     signif = SurrogatesSignificance(n = 1_000, tail = [:right, :right], rng = Xoshiro(1995))
     flags = significant_transitions(results, signif)
 

diff --git a/docs/src/examples/ks_paleojump.jl b/docs/src/examples/ks_paleojump.jl
@@ -48,7 +48,7 @@
 #    [Dansgaard-Oescher events and Critical Slowing Down](@ref) example)
 #    and set the appropriate time window.
 # 3. Define the function that estimates the change metric (i.e., the KS-statistic)
-# 3. Perform the sliding window analysis as in the [Tutorial](@ref) with [`estimate_indicator_changes`](@ref)
+# 3. Perform the sliding window analysis as in the [Tutorial](@ref) with [`estimate_changes`](@ref)
 # 4. Estimate the "confident" transitions in the data by comparing the estimated
 #    KS-statistic with a predefined threshold.
 
@@ -113,7 +113,7 @@ fig
 
 # ## Perform the sliding window analysis
 
-# This is just a straightforward call to [`estimate_indicator_changes`](@ref).
+# This is just a straightforward call to [`estimate_changes`](@ref).
 # In fact, it is even simpler than the tutorial. Here we skip completely
 # the "indicator" estimation step, and we evaluate the change metric directly
 # on input data. We do this by simply passing `nothing` as the indicators.
@@ -122,7 +122,7 @@ using TransitionsInTimeseries
 
 config = SlidingWindowConfig(nothing, normalized_KS_statistic; width_cha = 500)
 
-results = estimate_indicator_changes(config, xtrend, t)
+results = estimate_changes(config, xtrend, t)
 
 # Which we can visualize
 function visualize_results(results)
@@ -144,7 +144,7 @@ visualize_results(results)
 # The same thing happens if we alter the window duration
 
 config = SlidingWindowConfig(nothing, normalized_KS_statistic; width_cha = 200)
-results = estimate_indicator_changes(config, xtrend, t)
+results = estimate_changes(config, xtrend, t)
 visualize_results(results)
 
 # So one can easily obtain extra confidence by varying window

diff --git a/docs/src/examples/logistic.jl b/docs/src/examples/logistic.jl
@@ -76,7 +76,7 @@ config = SlidingWindowConfig(indicators, metric;
     width_ind, width_cha, stride_cha,
 )
 
-results = estimate_indicator_changes(config, x, rs)
+results = estimate_changes(config, x, rs)
 
 # Let's now plot the change metrics of the indicators
 

diff --git a/docs/src/tutorial.jl b/docs/src/tutorial.jl
@@ -185,7 +185,7 @@ Performing the step-by-step analysis of transition indicators is possible and mi
 
 TransitionsInTimeseries.jl wraps this typical workflow into a simple, extendable, and modular API that researchers can use with little effort. In addition, it allows performing the same analysis for several indicators / change metrics in one go.
 
-The interface is simple, and directly parallelizes the [Workflow](@ref workflow). It is based on the creation of a [`IndicatorsChangesConfig`](@ref), which contains a list of indicators, and corresponding metrics, to use for doing the above analysis. It also specifies what kind of surrogates to generate.
+The interface is simple, and directly parallelizes the [Workflow](@ref workflow). It is based on the creation of a [`ChangesConfig`](@ref), which contains a list of indicators, and corresponding metrics, to use for doing the above analysis. It also specifies what kind of surrogates to generate.
 
 ### Sliding windows
 
@@ -208,7 +208,7 @@ fig
 To perform all of the above analysis we follow a 2-step process.
 
 Step 1, we decide what indicators and change metrics to use in [`SlidingWindowConfig`](@ref) and apply those via
-a sliding window to the input timeseries using [`estimate_indicator_changes`](@ref).
+a sliding window to the input timeseries using [`estimate_changes`](@ref).
 =#
 
 ## These indicators are suitable for Critical Slowing Down
@@ -222,7 +222,7 @@ config = SlidingWindowConfig(indicators, change_metrics;
     width_ind = 400, width_cha = 30, whichtime = last)
 
 ## choices are processed
-results = estimate_indicator_changes(config, input, t)
+results = estimate_changes(config, input, t)
 
 #=
 We can conveniently plot the information contained in `results` by using
@@ -257,7 +257,7 @@ visualise the results conveniently:
 config = SegmentedWindowConfig(indicators, change_metrics,
     t[1:1], t[1200:1200]; whichtime = last, width_ind = 200,
     min_width_cha = 100)
-results = estimate_indicator_changes(config, input, t)
+results = estimate_changes(config, input, t)
 signif = SurrogatesSignificance(n = 1000, tail = [:right, :right])
 flags = significant_transitions(results, signif)
 fig = plot_changes_significance(results, signif)
diff --git a/docs/src/tutorial.md b/docs/src/tutorial.md
@@ -187,7 +187,7 @@ Performing the step-by-step analysis of transition indicators is possible and mi
 
 TransitionsInTimeseries.jl wraps this typical workflow into a simple, extendable, and modular API that researchers can use with little effort. In addition, it allows performing the same analysis for several indicators / change metrics in one go.
 
-The interface is simple, and directly parallelizes the [Workflow](@ref workflow). It is based on the creation of a [`IndicatorsChangesConfig`](@ref), which contains a list of indicators, and corresponding metrics, to use for doing the above analysis. It also specifies what kind of surrogates to generate.
+The interface is simple, and directly parallelizes the [Workflow](@ref workflow). It is based on the creation of a [`ChangesConfig`](@ref), which contains a list of indicators, and corresponding metrics, to use for doing the above analysis. It also specifies what kind of surrogates to generate.
 
 ### Sliding windows
 
@@ -210,7 +210,7 @@ fig
 To perform all of the above analysis we follow a 2-step process.
 
 Step 1, we decide what indicators and change metrics to use in [`SlidingWindowConfig`](@ref) and apply those via
-a sliding window to the input timeseries using [`estimate_indicator_changes`](@ref).
+a sliding window to the input timeseries using [`estimate_changes`](@ref).
 
 ````@example tutorial
 # These indicators are suitable for Critical Slowing Down
@@ -224,7 +224,7 @@ config = SlidingWindowConfig(indicators, change_metrics;
     width_ind = 400, width_cha = 30, whichtime = last)
 
 # choices are processed
-results = estimate_indicator_changes(config, input, t)
+results = estimate_changes(config, input, t)
 ````
 
 We can conveniently plot the information contained in `results` by using
@@ -260,7 +260,7 @@ visualise the results conveniently:
 config = SegmentedWindowConfig(indicators, change_metrics,
     t[1:1], t[1200:1200]; whichtime = last, width_ind = 200,
     min_width_cha = 100)
-results = estimate_indicator_changes(config, input, t)
+results = estimate_changes(config, input, t)
 signif = SurrogatesSignificance(n = 1000, tail = [:right, :right])
 flags = significant_transitions(results, signif)
 tv = plot_changes_significance(results, signif,

diff --git a/ext/TransitionVisualizations.jl b/ext/TransitionVisualizations.jl
@@ -6,10 +6,10 @@ using TransitionsInTimeseries, Makie
 const default_accent_linewidth = 3
 const default_colors = ["#7143E0", "#0A9A84", "#191E44", "#AF9327", "#701B80", "#2E6137"]
 
-default_indicator_label(res::IndicatorsChangesResults) = [shortname(
+default_indicator_label(res::ChangesResults) = [shortname(
     ind) for ind in res.config.indicators]
 
-default_chametric_label(res::IndicatorsChangesResults) = [shortname(
+default_chametric_label(res::ChangesResults) = [shortname(
     cha_metric) for cha_metric in res.config.change_metrics]
 
 shortname(metric) = string(nameof(metric))
@@ -85,7 +85,7 @@ function init_rowwise_visualisation(res, colors, indicator_names,
     raxs[end].xlabel = "Time"
     Makie.rowgap!(fig.layout, 10)
 
-    elements = [LineElement(color = (colors[1], transparency), linewidth = lw) for 
+    elements = [LineElement(color = (colors[1], transparency), linewidth = lw) for
         (lw, transparency) in [(accent_linewidth, 1), (1, 0.5)]]
     labels = ["original signal", "surro signals"]
     width = 0.5

diff --git a/src/TransitionsInTimeseries.jl b/src/TransitionsInTimeseries.jl
@@ -54,10 +54,10 @@ export ridgematrix, RidgeRegressionSlope, PrecomputedRidgeRegressionSlope
 export difference_of_means, difference_of_maxes
 
 # analysis
-export IndicatorsChangesConfig, SlidingWindowConfig, SegmentedWindowConfig
+export ChangesConfig, SlidingWindowConfig, SegmentedWindowConfig
 export SlidingWindowResults, SegmentedWindowResults
-export estimate_indicator_changes, IndicatorsChangesResults
-export TransitionsSignificance, significant_transitions, segmented_significance
+export estimate_changes, ChangesResults
+export Significance, significant_transitions, segmented_significance
 export ThresholdSignificance, QuantileSignificance, SigmaSignificance, SurrogatesSignificance
 
 # timeseries

diff --git a/src/analysis/api.jl b/src/analysis/api.jl
@@ -1,34 +1,37 @@
 """
-    IndicatorsChangesConfig
+    ChangesConfig
 
-Supertype used to define how indicators and their changes are estimated in
-[`estimate_indicator_changes`](@ref). Valid subtypes are:
+Supertype for how "changes" in a timeseries are estimated in [`estimate_changes`](@ref).
+"Changes" deemed statistically significant in [`significant_transitions`](@ref)
+are "transitions" in the timeseries.
+
+Existing subtypes of `ChangesConfig` are:
 
  - [`SlidingWindowConfig`](@ref).
  - [`SegmentedWindowConfig`](@ref).
 """
-abstract type IndicatorsChangesConfig end
+abstract type ChangesConfig end
 
 """
-    estimate_indicator_changes(config::IndicatorsChangesConfig, x [,t]) → result
+    estimate_changes(config::ChangesConfig, x [,t]) → result
 
 Estimate possible transitions for input timeseries `x` using the approach specified
-in the configuration type `config`, see [`IndicatorsChangesConfig`](@ref) for possibilities.
+in the configuration type `config`, see [`ChangesConfig`](@ref) for possibilities.
 `t` is the time vector corresponding to `x`, which defaults to `eachindex(x)`.
 
-Return the output as subtype of [`IndicatorsChangesResults`](@ref).
+Return the output as subtype of [`ChangesResults`](@ref).
 The particular form of the output depends on the `config` and is described in its docstring.
 Regardless of type, `result` can always be given to
 [`significant_transitions`](@ref) to deduce which possible transitions are statistically
 significant using a variety of significance tests.
 """
-function estimate_indicator_changes end
+function estimate_changes end
 # The function is extended via multiple dispatch in the specific files
 
 """
-    IndicatorsChangesResults
+    ChangesResults
 
-Supertype used to gather results of [`estimate_indicator_changes`](@ref).
+Supertype used to gather results of [`estimate_changes`](@ref).
 The concrete subtype instances are described in the docstrings of configuration types.
 """
-abstract type IndicatorsChangesResults end
+abstract type ChangesResults end
diff --git a/src/analysis/segmented_window.jl b/src/analysis/segmented_window.jl
@@ -2,7 +2,7 @@
     SegmentedWindowConfig <: IndicatorsChangeConfig
     SegmentedWindowConfig(indicators, change_metrics, tseg_start, tseg_end; kwargs...)
 
-A configuration that can be given to [`estimate_indicator_changes`](@ref).
+A configuration that can be given to [`estimate_changes`](@ref).
 It estimates transitions by estimating indicators and changes in user-defined
 window segments as follows:
 
@@ -23,7 +23,7 @@ The results output corresponding to `SlidingWindowConfig` is [`SegmentedWindowRe
 - `min_width_cha::Int=typemax(Int)`: minimal width required to perform the change metric estimation.
   If a segment is not sufficiently long, the change metric is `NaN`.
 """
-struct SegmentedWindowConfig{F, G, W<:Function} <: IndicatorsChangesConfig
+struct SegmentedWindowConfig{F, G, W<:Function} <: ChangesConfig
     indicators::F
     change_metrics::G
     tseg_start::Vector
@@ -62,7 +62,7 @@ function SegmentedWindowConfig(
     )
 end
 
-function estimate_indicator_changes(config::SegmentedWindowConfig, x, t)
+function estimate_changes(config::SegmentedWindowConfig, x, t)
     X, T = eltype(x), eltype(t)
     (; indicators, change_metrics, tseg_start, tseg_end) = config
     n_ind = length(indicators)
@@ -122,9 +122,9 @@ function segment_time(t::AbstractVector, t1, t2)
 end
 
 """
-    SegmentedWindowResults <: IndicatorsChangesResults
+    SegmentedWindowResults <: ChangesResults
 
-A struct containing the output of [`estimate_indicator_changes`](@ref) used with
+A struct containing the output of [`estimate_changes`](@ref) used with
 [`SegmentedWindowConfig`](@ref). It can be used for further analysis, visualization,
 or given to [`significant_transitions`](@ref).
 
@@ -148,7 +148,7 @@ It has the following fields that the user may access
 - `precomp_change_metrics` vector containing the precomputed change metrics of each segment.
 """
 struct SegmentedWindowResults{TT, T<:Real, X<:Real, XX<:AbstractVector{X},
-    W, Z} <: IndicatorsChangesResults
+    W, Z} <: ChangesResults
     t::TT # original time vector; most often it is `Base.OneTo`.
     x::XX
     t_indicator::Vector{Vector{T}}