diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 5e62ca9..24014a9 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.4","generation_timestamp":"2024-07-13T14:44:16","documenter_version":"1.5.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.4","generation_timestamp":"2024-07-16T08:14:29","documenter_version":"1.5.0"}} \ No newline at end of file diff --git a/dev/api/index.html b/dev/api/index.html index 5506200..281815f 100644 --- a/dev/api/index.html +++ b/dev/api/index.html @@ -1,18 +1,18 @@ -API · TransitionsInTimeseries.jl

API

Main analysis functions

TransitionsInTimeseries.estimate_changesFunction
estimate_changes(config::ChangesConfig, x [,t]) → result

Estimate possible transitions for input timeseries x using the approach specified in the configuration type config, see ChangesConfig for possibilities. t is the time vector corresponding to x, which defaults to eachindex(x).

Return the output as subtype of ChangesResults. The particular form of the output depends on the config and is described in its docstring. Regardless of type, result can always be given to significant_transitions to deduce which possible transitions are statistically significant using a variety of significance tests.

source

Sliding window

TransitionsInTimeseries.SlidingWindowConfigType
SlidingWindowConfig <: ChangesConfig
+API · TransitionsInTimeseries.jl

API

Main analysis functions

TransitionsInTimeseries.estimate_changesFunction
estimate_changes(config::ChangesConfig, x [,t]) → result

Estimate possible transitions for input timeseries x using the approach specified in the configuration type config, see ChangesConfig for possibilities. t is the time vector corresponding to x, which defaults to eachindex(x).

Return the output as subtype of ChangesResults. The particular form of the output depends on the config and is described in its docstring. Regardless of type, result can always be given to significant_transitions to deduce which possible transitions are statistically significant using a variety of significance tests.

source

Sliding window

TransitionsInTimeseries.SlidingWindowConfigType
SlidingWindowConfig <: ChangesConfig
 SlidingWindowConfig(indicators, change_metrics; kwargs...)

A configuration that can be given to estimate_changes. It estimates transitions by a sliding window approach:

  1. Estimate the timeseries of an indicator by sliding a window over the input timeseries.
  2. Estimate changes of an indicator by sliding a window of the change metric over the indicator timeseries.

Both indicators and change metrics are generic Julia functions that input an x::AbstractVector and output an s::Real. Any function may be given and see making custom indicators/change metrics in the documentation for more information on possible optimizations.

indicators can be a single function or a tuple of indicators. Similarly, change_metrics can be a tuple or a single function. If tuples, the length of indicators and change_metrics must match. This way the analysis can be efficiently repeated for many indicators and/or change metrics.

The results output corresponding to SlidingWindowConfig is SlidingWindowResults.

Step 1. is skipped if nothing is provided as indicators, in which case the change metrics are estimated directly from input data.

Keyword arguments

  • width_ind::Int=100, stride_ind::Int=1: width and stride given to WindowViewer to compute the indicator from the input timeseries.
  • width_cha::Int=50, stride_cha::Int=1: width and stride given to WindowViewer to compute the change metric timeseries from the indicator timeseries.
  • whichtime = midpoint: The time vector corresponding to the indicators / change metric timeseries is obtained from t in estimate_changes using the keyword whichtime. Options include:
    • last: use the last timepoint of each window
    • midpoint: use the mid timepoint of each time window
    • first: use first timepoint of each window
    In fact, the indicators time vector is computed simply via
    t_indicator = windowmap(whichtime, t; width_ind, stride_ind)
    -t_change = windowmap(whichtime, t_indicator; width_cha, stride_cha)
    so any other function of the time window may be given to extract the time point itself, such as mean or median.
  • T = Float64: Element type of input timeseries to initialize some computations.
source
TransitionsInTimeseries.SlidingWindowResultsType
SlidingWindowResults <: ChangesResults

A struct containing the output of estimate_changes used with SlidingWindowConfig. It can be used for further analysis, visualization, or given to significant_transitions.

It has the following fields that the user may access

  • x: the input timeseries.

  • t: the time vector of the input timeseries.

  • x_indicator, the indicator timeseries (matrix with each column one indicator).

  • t_indicator, the time vector of the indicator timeseries.

  • x_change, the change metric timeseries (matrix with each column one change metric).

  • t_change, the time vector of the change metric timeseries.

  • config::SlidingWindowConfig, what was used for the analysis.

source

Segmented window

TransitionsInTimeseries.SegmentedWindowConfigType
SegmentedWindowConfig <: IndicatorsChangeConfig
-SegmentedWindowConfig(indicators, change_metrics, tseg_start, tseg_end; kwargs...)

A configuration that can be given to estimate_changes. It estimates transitions by estimating indicators and changes in user-defined window segments as follows:

  1. For each segment specified, estimate the corresponding indicator timeseries by sliding a window over the input timeseries (within the window segment).
  2. For each segment of the indicator timeseries, estimate a scalar change metric by applying the change metric over the full segment of the indicator timeseries.d

tseg_start, tseg_end are the starts and ends of the window segments (the window segments may overlap, that's okay). indicators, change_metrics are identical as in SlidingWindowConfig.

The results output corresponding to SlidingWindowConfig is SegmentedWindowResults.

Keyword arguments

  • width_ind::Int=100, stride_ind::Int=1, whichtime = midpoint, T = Float64: keywords identical as in SlidingWindowConfig.
  • min_width_cha::Int=typemax(Int): minimal width required to perform the change metric estimation. If a segment is not sufficiently long, the change metric is NaN.
source
TransitionsInTimeseries.SegmentedWindowResultsType
SegmentedWindowResults <: ChangesResults

A struct containing the output of estimate_changes used with SegmentedWindowConfig. It can be used for further analysis, visualization, or given to significant_transitions.

It has the following fields that the user may access

  • x: the input timeseries.

  • t: the time vector of the input timeseries.

  • x_indicator::Vector{Matrix}, with x_indicator[k] the indicator timeseries (matrix with each column one indicator) of the k-th segment.

  • t_indicator::Vector{Vector}, with t_indicator[k] the time vector of the indicator timeseries for the k-th segment.

  • x_change::Matrix, the change metric values with x[k, i] the change metric of the i-th indicator for the k-th segment.

  • t_change, the time vector of the change metric.

  • config::SegmentedWindowConfig, what was used for the analysis.

  • i1::Vector{Int} indices corresponding to start time of each segment.

  • i2::Vector{Int} indices corresponding to end time of each segment.

  • precomp_change_metrics vector containing the precomputed change metrics of each segment.

source

Slope change

TransitionsInTimeseries.SlopeChangeConfigType
SlopeChangeConfig <: ChangesConfig
-SlopeChangeConfig(; indicator = nothing, kw...)

A configuration that can be given to estimate_changes. It estimates a change of slope in the timeseries by fitting two connected linear segments to the timeseries, returning the results (i.e., the two-linear fits) as SlopeChangeResults.

Keyword arguments

  • indicator = nothing: if not nothing. Otherwise it should be a function f(x) -> Real. The slope fitting is then done over an indicator of the timeseries, which itself is estimated via a sliding window exactly as in SlidingWindowConfig.
  • width_ind, stride_ind, whichtime: exactly as in SlidingWindowConfig if indicator is not nothing.
source
TransitionsInTimeseries.SlopeChangeResultsType
SlopeChangeResults <: ChangesResults

A struct containing the output of estimate_changes used with SlopeChangeConfig. It can be used for further analysis, visualization, or given to significant_transitions. The only significance type that you can use this with significant_transitions is SlopeChangeSignificance.

It has the following fields that the user may access:

  • x: the input timeseries.
  • t: the time vector of the input timeseries.
  • x_indicator, the indicator timeseries.
  • t_indicator, the time vector of the indicator timeseries.
  • t_change, the time the slope changes.
  • fitparams = a, b, c, d, the fitted linear coefficients, a + b*t before. t_change and c + d*t after t_change.
source

Significance testing

TransitionsInTimeseries.significant_transitionsFunction
significant_transitions(res::ChangesResults, signif::Significance)

Estimate significant transtions in res using the method described by signif. Return flags, a Boolean matrix with identical size as the changes stored in res (which typically is stored in the field res.x_change). flags is true wherever a change metric of res is deemed significant.

source
TransitionsInTimeseries.SurrogatesSignificanceType
SurrogatesSignificance <: Significance
-SurrogatesSignificance(; kwargs...)

A configuration struct for significance testing significant_transitions using timeseries surrogates.

Keyword arguments

  • surromethod = RandomFourier(): method to generate surrogates
  • n = 1000: how many surrogates to generate
  • rng = Random.default_rng(): random number generator for the surrogates
  • p = 0.05: threshold for significance of the p-value
  • tail = :both: tail type used, see below

Description

When used with ChangesResults, significance is estimated as follows: n surrogates from the input timeseries are generated using surromethod, which is any Surrogate subtype provided by TimeseriesSurrogates.jl. For each surrogate, the indicator and then change metric timeseries is extracted. The values of the surrogate change metrics form a distribution of values (one at each time point). The value of the original change metric is compared to that of the surrogate distribution and a p-value is extracted according to the specified tail. The p-value is compared with p to claim significance. After using SurrogatesSignificance, you may access the full p-values before thresholding in the field .pvalues (to e.g., threshold with different p).

The p-value is simply the proportion of surrogate change metric values that exceed (for tail = :right) or subseed (tail = :left) the original change metric at each given time point. Use tail = :left if the surrogate data are expected to have higher change metric values. This is the case for statistics that quantify entropy. For statistics that quantify autocorrelation, use tail = :right instead. For anything else, use the default tail = :both. An iterable of tail values can also be given, in which case a specific tail is used for each change metric in ChangesResults.

Note that the raw p-values can be accessed in the field .pvalues, after calling the significant_transitions function with SurrogatesSignificance, in case you wish to obtain a different threshold of the p-values.

source
TransitionsInTimeseries.ThresholdSignificanceType
ThresholdSignificance(threshold::Real; tail = :right) <: Significance

A configuration struct for significance testing in significant_transitions. Significance is estimated by comparing the value of each change metric with the given threshold. Values that exceed the threshold (if tail = :right) or subseed the threshold (if tail = :left) are deemed significant. If tail = :both then either condition is checked.

source
TransitionsInTimeseries.SigmaSignificanceType
SigmaSignificance(; factor = 3.0, tail = :both) <: Significance

A configuration struct for significance testing significant_transitions. When used with ChangesResults, significance is estimated by comparing how many standard deviations (σ) the value exceeds the mean value (μ). Values that exceed (if tail = :right) μ + factor*σ, or subseed (if tail = :left) μ - factor*σ are deemed significant. If tail = :both then either condition is checked.

factor can also be a vector of values, in which case a different value is used for each change metric.

See also QuantileSignificance.

source
TransitionsInTimeseries.QuantileSignificanceType
QuantileSignificance(; p = 0.95, tail = :both) <: Significance

A configuration struct for significance testing significant_transitions. When used with ChangesResults, significance is estimated by comparing the value of each change metric with its p-quantile. Values that exceed the p-quantile (if tail = :right) or subseed the 1-p-quantile (if tail = :left) are deemed significant. If tail = :both then either condition is checked.

QuantileSignficance guarantees that some values will be significant by the very definition of what a quantile is. See also SigmaSignificance that is similar but does not have this guarantee.

source
TransitionsInTimeseries.SlopeChangeSignificanceType
SlopeChangeSignificance(; moe_slope, moe_offset, slope_diff = moe_slope, pvalue = 0.05)

Test whether the result of SlopeChangeResults is statistically significant.

Two tests are done:

  1. Check whether the margin of error of the fitted parameters a, b, c, d of the two linear segments a + b*t, c + d*t is less than the specified margins of error, for a chosen pvalue.
  2. Test that the two slopes b, d have difference greater than slope_diff.

The Boolean & of the above two is the final test.

The margin of error is simply half the size of the confidence interval, also known as radius of the confidence interval.

source

Indicators

Note that any Julia function can be an indicator or change metric, so the list here is only just a couple of indicators directly implemented in this package.

Value distribution

Statistics.meanFunction
mean(A::AbstractArray, w::AbstractWeights[, dims::Int])

Compute the weighted mean of array A with weight vector w (of type AbstractWeights). If dim is provided, compute the weighted mean along dimension dims.

Examples

n = 20
+t_change = windowmap(whichtime, t_indicator; width_cha, stride_cha)
so any other function of the time window may be given to extract the time point itself, such as mean or median.
  • T = Float64: Element type of input timeseries to initialize some computations.
  • source
    TransitionsInTimeseries.SlidingWindowResultsType
    SlidingWindowResults <: ChangesResults

    A struct containing the output of estimate_changes used with SlidingWindowConfig. It can be used for further analysis, visualization, or given to significant_transitions.

    It has the following fields that the user may access

    • x: the input timeseries.

    • t: the time vector of the input timeseries.

    • x_indicator, the indicator timeseries (matrix with each column one indicator).

    • t_indicator, the time vector of the indicator timeseries.

    • x_change, the change metric timeseries (matrix with each column one change metric).

    • t_change, the time vector of the change metric timeseries.

    • config::SlidingWindowConfig, what was used for the analysis.

    source

    Segmented window

    TransitionsInTimeseries.SegmentedWindowConfigType
    SegmentedWindowConfig <: IndicatorsChangeConfig
    +SegmentedWindowConfig(indicators, change_metrics, tseg_start, tseg_end; kwargs...)

    A configuration that can be given to estimate_changes. It estimates transitions by estimating indicators and changes in user-defined window segments as follows:

    1. For each segment specified, estimate the corresponding indicator timeseries by sliding a window over the input timeseries (within the window segment).
    2. For each segment of the indicator timeseries, estimate a scalar change metric by applying the change metric over the full segment of the indicator timeseries.d

    tseg_start, tseg_end are the starts and ends of the window segments (the window segments may overlap, that's okay). indicators, change_metrics are identical as in SlidingWindowConfig.

    The results output corresponding to SlidingWindowConfig is SegmentedWindowResults.

    Keyword arguments

    • width_ind::Int=100, stride_ind::Int=1, whichtime = midpoint, T = Float64: keywords identical as in SlidingWindowConfig.
    • min_width_cha::Int=typemax(Int): minimal width required to perform the change metric estimation. If a segment is not sufficiently long, the change metric is NaN.
    source
    TransitionsInTimeseries.SegmentedWindowResultsType
    SegmentedWindowResults <: ChangesResults

    A struct containing the output of estimate_changes used with SegmentedWindowConfig. It can be used for further analysis, visualization, or given to significant_transitions.

    It has the following fields that the user may access

    • x: the input timeseries.

    • t: the time vector of the input timeseries.

    • x_indicator::Vector{Matrix}, with x_indicator[k] the indicator timeseries (matrix with each column one indicator) of the k-th segment.

    • t_indicator::Vector{Vector}, with t_indicator[k] the time vector of the indicator timeseries for the k-th segment.

    • x_change::Matrix, the change metric values with x[k, i] the change metric of the i-th indicator for the k-th segment.

    • t_change, the time vector of the change metric.

    • config::SegmentedWindowConfig, what was used for the analysis.

    • i1::Vector{Int} indices corresponding to start time of each segment.

    • i2::Vector{Int} indices corresponding to end time of each segment.

    • precomp_change_metrics vector containing the precomputed change metrics of each segment.

    source

    Slope change

    TransitionsInTimeseries.SlopeChangeConfigType
    SlopeChangeConfig <: ChangesConfig
    +SlopeChangeConfig(; indicator = nothing, kw...)

    A configuration that can be given to estimate_changes. It estimates a change of slope in the timeseries by fitting two connected linear segments to the timeseries, returning the results (i.e., the two-linear fits) as SlopeChangeResults.

    Keyword arguments

    • indicator = nothing: if not nothing. Otherwise it should be a function f(x) -> Real. The slope fitting is then done over an indicator of the timeseries, which itself is estimated via a sliding window exactly as in SlidingWindowConfig.
    • width_ind, stride_ind, whichtime: exactly as in SlidingWindowConfig if indicator is not nothing.
    source
    TransitionsInTimeseries.SlopeChangeResultsType
    SlopeChangeResults <: ChangesResults

    A struct containing the output of estimate_changes used with SlopeChangeConfig. It can be used for further analysis, visualization, or given to significant_transitions. The only significance type that you can use this with significant_transitions is SlopeChangeSignificance.

    It has the following fields that the user may access:

    • x: the input timeseries.
    • t: the time vector of the input timeseries.
    • x_indicator, the indicator timeseries.
    • t_indicator, the time vector of the indicator timeseries.
    • t_change, the time the slope changes.
    • fitparams = a, b, c, d, the fitted linear coefficients, a + b*t before. t_change and c + d*t after t_change.
    source

    Significance testing

    TransitionsInTimeseries.significant_transitionsFunction
    significant_transitions(res::ChangesResults, signif::Significance)

    Estimate significant transtions in res using the method described by signif. Return flags, a Boolean matrix with identical size as the changes stored in res (which typically is stored in the field res.x_change). flags is true wherever a change metric of res is deemed significant.

    source
    TransitionsInTimeseries.SurrogatesSignificanceType
    SurrogatesSignificance <: Significance
    +SurrogatesSignificance(; kwargs...)

    A configuration struct for significance testing significant_transitions using timeseries surrogates.

    Keyword arguments

    • surromethod = RandomFourier(): method to generate surrogates
    • n = 1000: how many surrogates to generate
    • rng = Random.default_rng(): random number generator for the surrogates
    • p = 0.05: threshold for significance of the p-value
    • tail = :both: tail type used, see below

    Description

    When used with ChangesResults, significance is estimated as follows: n surrogates from the input timeseries are generated using surromethod, which is any Surrogate subtype provided by TimeseriesSurrogates.jl. For each surrogate, the indicator and then change metric timeseries is extracted. The values of the surrogate change metrics form a distribution of values (one at each time point). The value of the original change metric is compared to that of the surrogate distribution and a p-value is extracted according to the specified tail. The p-value is compared with p to claim significance. After using SurrogatesSignificance, you may access the full p-values before thresholding in the field .pvalues (to e.g., threshold with different p).

    The p-value is simply the proportion of surrogate change metric values that exceed (for tail = :right) or subseed (tail = :left) the original change metric at each given time point. Use tail = :left if the surrogate data are expected to have higher change metric values. This is the case for statistics that quantify entropy. For statistics that quantify autocorrelation, use tail = :right instead. For anything else, use the default tail = :both. An iterable of tail values can also be given, in which case a specific tail is used for each change metric in ChangesResults.

    Note that the raw p-values can be accessed in the field .pvalues, after calling the significant_transitions function with SurrogatesSignificance, in case you wish to obtain a different threshold of the p-values.

    source
    TransitionsInTimeseries.ThresholdSignificanceType
    ThresholdSignificance(threshold::Real; tail = :right) <: Significance

    A configuration struct for significance testing in significant_transitions. Significance is estimated by comparing the value of each change metric with the given threshold. Values that exceed the threshold (if tail = :right) or subseed the threshold (if tail = :left) are deemed significant. If tail = :both then either condition is checked.

    source
    TransitionsInTimeseries.SigmaSignificanceType
    SigmaSignificance(; factor = 3.0, tail = :both) <: Significance

    A configuration struct for significance testing significant_transitions. When used with ChangesResults, significance is estimated by comparing how many standard deviations (σ) the value exceeds the mean value (μ). Values that exceed (if tail = :right) μ + factor*σ, or subseed (if tail = :left) μ - factor*σ are deemed significant. If tail = :both then either condition is checked.

    factor can also be a vector of values, in which case a different value is used for each change metric.

    See also QuantileSignificance.

    source
    TransitionsInTimeseries.QuantileSignificanceType
    QuantileSignificance(; p = 0.95, tail = :both) <: Significance

    A configuration struct for significance testing significant_transitions. When used with ChangesResults, significance is estimated by comparing the value of each change metric with its p-quantile. Values that exceed the p-quantile (if tail = :right) or subseed the 1-p-quantile (if tail = :left) are deemed significant. If tail = :both then either condition is checked.

    QuantileSignficance guarantees that some values will be significant by the very definition of what a quantile is. See also SigmaSignificance that is similar but does not have this guarantee.

    source
    TransitionsInTimeseries.SlopeChangeSignificanceType
    SlopeChangeSignificance(; moe_slope, moe_offset, slope_diff = moe_slope, pvalue = 0.05)

    Test whether the result of SlopeChangeResults is statistically significant.

    Two tests are done:

    1. Check whether the margin of error of the fitted parameters a, b, c, d of the two linear segments a + b*t, c + d*t is less than the specified margins of error, for a chosen pvalue.
    2. Test that the two slopes b, d have difference greater than slope_diff.

    The Boolean & of the above two is the final test.

    The margin of error is simply half the size of the confidence interval, also known as radius of the confidence interval.

    source

    Indicators

    Note that any Julia function can be an indicator or change metric, so the list here is only just a couple of indicators directly implemented in this package.

    Value distribution

    Statistics.meanFunction
    mean(A::AbstractArray, w::AbstractWeights[, dims::Int])

    Compute the weighted mean of array A with weight vector w (of type AbstractWeights). If dim is provided, compute the weighted mean along dimension dims.

    Examples

    n = 20
     x = rand(n)
     w = rand(n)
    -mean(x, weights(w))
    source
    StatsBase.skewnessFunction
    skewness(v, [wv::AbstractWeights], m=mean(v))

    Compute the standardized skewness of a real-valued array v, optionally specifying a weighting vector wv and a center m.

    source
    StatsBase.kurtosisFunction
    kurtosis(v, [wv::AbstractWeights], m=mean(v))

    Compute the excess kurtosis of a real-valued array v, optionally specifying a weighting vector wv and a center m.

    source

    Critical Slowing Down

    Statistics.varFunction
    var(x::AbstractArray, w::AbstractWeights, [dim]; mean=nothing, corrected=false)

    Compute the variance of a real-valued array x, optionally over a dimension dim. Observations in x are weighted using weight vector w. The uncorrected (when corrected=false) sample variance is defined as:

    \[\frac{1}{\sum{w}} \sum_{i=1}^n {w_i\left({x_i - μ}\right)^2 }\]

    where $n$ is the length of the input and $μ$ is the mean. The unbiased estimate (when corrected=true) of the population variance is computed by replacing $\frac{1}{\sum{w}}$ with a factor dependent on the type of weights used:

    • AnalyticWeights: $\frac{1}{\sum w - \sum {w^2} / \sum w}$
    • FrequencyWeights: $\frac{1}{\sum{w} - 1}$
    • ProbabilityWeights: $\frac{n}{(n - 1) \sum w}$ where $n$ equals count(!iszero, w)
    • Weights: ArgumentError (bias correction not supported)
    source
    var(ce::CovarianceEstimator, x::AbstractVector; mean=nothing)

    Compute the variance of the vector x using the estimator ce.

    source
    TransitionsInTimeseries.ar1_whitenoiseFunction
    ar1_whitenoise(x::AbstractVector)

    Return the AR1 regression coefficient of a time series x by computing the analytic solution of the least-square parameter estimation under white-noise assumption for the data-generating process:

    \[\theta = \sum_{i=2}^{n} x_i x_{i-1} / \sum_{i=2}^{n} x_{i-1}^2\]

    source

    Spectrum

    TransitionsInTimeseries.LowfreqPowerSpectrumType
    LowfreqPowerSpectrum(; q_lofreq = 0.1)

    Return a PrecomputableFunction containing all the necessary fields to generate a PrecomputedLowfreqPowerSpectrum. The latter can be initialized by precompute:

    lfps = precompute( LowfreqPowerSpectrum() )

    Keyword arguments:

    • q_lofreq: a number between 0 and 1 that characterises which portion of the

    frequency spectrum is considered to be low. For instance, q_lofreq = 0.1 implies that the lowest 10% of frequencies are considered to be the low ones.

    source
    TransitionsInTimeseries.PrecomputedLowfreqPowerSpectrumType
    PrecomputedLowfreqPowerSpectrum

    A struct containing all the precomputed fields to efficiently perform repetitive computation of the low-frequency power spectrum (LFPS), a number between 0 and 1 that characterizes the amount of power contained in the low frequencies of the power density spectrum of x. Once lfps::PrecomputedLowfreqPowerSpectrum is initialized, it can be used as a function to obtain the LFPS of x::AbstractVector by:

    lfps(x)
    source

    Nonlinear dynamics

    Indicators that come from nonlinear timeseries analysis typically quantify some entropy-like or complexity measure from the timeseries. Thousands (literally) such measures are provided out of the box by the ComplexityMeasures.jl package. Given that any of these may be used as an indicator or change metric, we made the decision to not copy-paste any measure here, as it is easy for the user to use any of them.

    For example, using the permutation entropy as an indicator is as simple as doing

    using ComplexityMeasures
    +mean(x, weights(w))
    source
    StatsBase.skewnessFunction
    skewness(v, [wv::AbstractWeights], m=mean(v))

    Compute the standardized skewness of a real-valued array v, optionally specifying a weighting vector wv and a center m.

    source
    StatsBase.kurtosisFunction
    kurtosis(v, [wv::AbstractWeights], m=mean(v))

    Compute the excess kurtosis of a real-valued array v, optionally specifying a weighting vector wv and a center m.

    source

    Critical Slowing Down

    Statistics.varFunction
    var(x::AbstractArray, w::AbstractWeights, [dim]; mean=nothing, corrected=false)

    Compute the variance of a real-valued array x, optionally over a dimension dim. Observations in x are weighted using weight vector w. The uncorrected (when corrected=false) sample variance is defined as:

    \[\frac{1}{\sum{w}} \sum_{i=1}^n {w_i\left({x_i - μ}\right)^2 }\]

    where $n$ is the length of the input and $μ$ is the mean. The unbiased estimate (when corrected=true) of the population variance is computed by replacing $\frac{1}{\sum{w}}$ with a factor dependent on the type of weights used:

    • AnalyticWeights: $\frac{1}{\sum w - \sum {w^2} / \sum w}$
    • FrequencyWeights: $\frac{1}{\sum{w} - 1}$
    • ProbabilityWeights: $\frac{n}{(n - 1) \sum w}$ where $n$ equals count(!iszero, w)
    • Weights: ArgumentError (bias correction not supported)
    source
    var(ce::CovarianceEstimator, x::AbstractVector; mean=nothing)

    Compute the variance of the vector x using the estimator ce.

    source
    TransitionsInTimeseries.ar1_whitenoiseFunction
    ar1_whitenoise(x::AbstractVector)

    Return the AR1 regression coefficient of a time series x by computing the analytic solution of the least-square parameter estimation under white-noise assumption for the data-generating process:

    \[\theta = \sum_{i=2}^{n} x_i x_{i-1} / \sum_{i=2}^{n} x_{i-1}^2\]

    source

    Spectrum

    TransitionsInTimeseries.LowfreqPowerSpectrumType
    LowfreqPowerSpectrum(; q_lofreq = 0.1)

    Return a PrecomputableFunction containing all the necessary fields to generate a PrecomputedLowfreqPowerSpectrum. The latter can be initialized by precompute:

    lfps = precompute( LowfreqPowerSpectrum() )

    Keyword arguments:

    • q_lofreq: a number between 0 and 1 that characterises which portion of the

    frequency spectrum is considered to be low. For instance, q_lofreq = 0.1 implies that the lowest 10% of frequencies are considered to be the low ones.

    source
    TransitionsInTimeseries.PrecomputedLowfreqPowerSpectrumType
    PrecomputedLowfreqPowerSpectrum

    A struct containing all the precomputed fields to efficiently perform repetitive computation of the low-frequency power spectrum (LFPS), a number between 0 and 1 that characterizes the amount of power contained in the low frequencies of the power density spectrum of x. Once lfps::PrecomputedLowfreqPowerSpectrum is initialized, it can be used as a function to obtain the LFPS of x::AbstractVector by:

    lfps(x)
    source

    Nonlinear dynamics

    Indicators that come from nonlinear timeseries analysis typically quantify some entropy-like or complexity measure from the timeseries. Thousands (literally) such measures are provided out of the box by the ComplexityMeasures.jl package. Given that any of these may be used as an indicator or change metric, we made the decision to not copy-paste any measure here, as it is easy for the user to use any of them.

    For example, using the permutation entropy as an indicator is as simple as doing

    using ComplexityMeasures
     est = OrdinalPatterns(; m = 3) # order 3
     # create a function that given timeseries returns permutation entropy
    -indicator = x -> entropy_normalized(est, x)

    and giving the created indicator to e.g., SlidingWindowConfig.

    Change metrics

    Slope

    TransitionsInTimeseries.kendalltauFunction
    kendalltau(x::AbstractVector)

    Compute the kendall-τ correlation coefficient of the time series x. kendalltau can be used as a change metric focused on trend.

    source
    TransitionsInTimeseries.spearmanFunction
    spearman(x::AbstractVector)

    Compute the spearman correlation coefficient of the time series x. spearman can be used as a change metric focused on trend.

    source

    Value distribution differences

    TransitionsInTimeseries.difference_of_meansFunction
    difference_of_means(x::AbstractArray)

    Return the absolute difference of the means of the first and second halfs of x. difference_of_means can be used as a change metric focused on value differences. Creating similar statistical differences using other moments instead of mean is trivial. In fact, the source of difference_of_means is just:

    # assumes 1-based indexing
    +indicator = x -> entropy_normalized(est, x)

    and giving the created indicator to e.g., SlidingWindowConfig.

    Change metrics

    Slope

    TransitionsInTimeseries.kendalltauFunction
    kendalltau(x::AbstractVector)

    Compute the kendall-τ correlation coefficient of the time series x. kendalltau can be used as a change metric focused on trend.

    source
    TransitionsInTimeseries.spearmanFunction
    spearman(x::AbstractVector)

    Compute the spearman correlation coefficient of the time series x. spearman can be used as a change metric focused on trend.

    source

    Value distribution differences

    TransitionsInTimeseries.difference_of_meansFunction
    difference_of_means(x::AbstractArray)

    Return the absolute difference of the means of the first and second halfs of x. difference_of_means can be used as a change metric focused on value differences. Creating similar statistical differences using other moments instead of mean is trivial. In fact, the source of difference_of_means is just:

    # assumes 1-based indexing
     n = length(x)
     x1 = view(x, 1:n÷2)
     x2 = view(x, (n÷2 + 1):n)
    -return abs(mean(x1) - mean(x2))

    difference_of_means can also sensibly be used for windows of size 2, in which case the change metric timeseries is the same as the abs.(diff(...)) of the indicator timeseries.

    source

    Make your own indicator/metric!

    The only difference between what is an "indicator" and what is a "change metric" is purely conceptual. As far as the code base of TransitionsInTimeseries.jl is concerned, they are both functions f: x::AbstractVector{Real} -> f(x)::Real. As a user you may give any such function for an indicator or change metric.

    There are situations where you may optimize such a function based on knowledge of input x type and length, in which case you may use PrecomputableFunction:

    Surrogates

    For the surrogate generation, you can use any subtype of Surrogate defined in Timeseriessurrogates.jl.

    Sliding windows

    TransitionsInTimeseries.WindowViewerType
    WindowViewer(x; width, stride)

    Initialize an iterator that generates views over the given timeseries x based on a window with a given width, incrementing the window views with the given stride. You can use this directly with map, such as map(std, WindowViewer(x, ...)) would give you the moving-window-timeseries of the std of x.

    If not given, the keywords width, stride are respectively taken as default_window_width(x) and 1.

    source
    TransitionsInTimeseries.windowmapFunction
    windowmap(f::Function, x::AbstractVector; kwargs...) → mapped_f

    A shortcut for first generating a wv = WindowViewer(x; kwargs...) and then applying mapped_f = map(f, wv). If x is accompanied by a time vector t, you probably also want to call this function with t instead of x and with one of mean, midpoint, midvalue as f to obtain a time vector for the mapped_f output.

    source

    Load data

    Visualization

    TransitionsInTimeseries.plot_indicator_changesFunction
    plot_indicator_changes(res) → (fig, axs)

    Return fig::Figure and axs::Matrix{Axis}, on which res::ChangesResults has been visualised.

    Keyword arguments:

    • colors = default_colors sets the colors of the line plots that are to

    be cycled through. The default correspond to the color scheme of Julia Dynamics.

    • `indicatornames = defaultindicatorlabel(res), chametricnames =

    defaultchametriclabel(res)sets the labels for the indicators and the change metrics, with the default inferring them from the names ofres.config.indicatorsandres.config.change_metrics`.

    • accent_linewidth = 3 sets the line width for the original signals (the

    surrogates have linewidth = 1)

    source
    TransitionsInTimeseries.plot_significance!Function
    plot_significance!(axs, res, signif)

    Update the axs::Matrix{Axis} of a figure obtained with plot_indicator_changes(res) with the signif::SurrogatesSignificance.

    Keyword arguments:

    • flags = nothing provides the significance flags, for instance obtained by

    thresholding the pvalues.

    • nsurro = 20 sets the number of surrogates to plot.
    source

    Utils

    default_window_width
    +return abs(mean(x1) - mean(x2))

    difference_of_means can also sensibly be used for windows of size 2, in which case the change metric timeseries is the same as the abs.(diff(...)) of the indicator timeseries.

    source

    Make your own indicator/metric!

    The only difference between what is an "indicator" and what is a "change metric" is purely conceptual. As far as the code base of TransitionsInTimeseries.jl is concerned, they are both functions f: x::AbstractVector{Real} -> f(x)::Real. As a user you may give any such function for an indicator or change metric.

    There are situations where you may optimize such a function based on knowledge of input x type and length, in which case you may use PrecomputableFunction:

    TransitionsInTimeseries.PrecomputableFunctionType
    PrecomputableFunction

    Supertype of structs containing the necessary field to precompute a ::Function by:

    precompute(f::PrecomputableFunction, t)
    source
    TransitionsInTimeseries.precomputeFunction
    precompute(f, t)

    Precompute the function f given a time vector t.

    source

    Surrogates

    For the surrogate generation, you can use any subtype of Surrogate defined in Timeseriessurrogates.jl.

    Sliding windows

    TransitionsInTimeseries.WindowViewerType
    WindowViewer(x; width, stride)

    Initialize an iterator that generates views over the given timeseries x based on a window with a given width, incrementing the window views with the given stride. You can use this directly with map, such as map(std, WindowViewer(x, ...)) would give you the moving-window-timeseries of the std of x.

    If not given, the keywords width, stride are respectively taken as default_window_width(x) and 1.

    source
    TransitionsInTimeseries.windowmapFunction
    windowmap(f::Function, x::AbstractVector; kwargs...) → mapped_f

    A shortcut for first generating a wv = WindowViewer(x; kwargs...) and then applying mapped_f = map(f, wv). If x is accompanied by a time vector t, you probably also want to call this function with t instead of x and with one of mean, midpoint, midvalue as f to obtain a time vector for the mapped_f output.

    source
    TransitionsInTimeseries.windowmap!Function
    windowmap!(f::Function, out, x::AbstractVector; kwargs...)

    Same as windowmap, but writes the output in-place in out.

    source

    Load data

    TransitionsInTimeseries.load_linear_vs_doublewellMethod
    load_linear_vs_doublewell()

    Load prototypical data from a linear and a double-well model to test some indicators.

    source

    Visualization

    TransitionsInTimeseries.plot_indicator_changesFunction
    plot_indicator_changes(res) → (fig, axs)

    Return fig::Figure and axs::Matrix{Axis}, on which res::ChangesResults has been visualised.

    Keyword arguments:

    • colors = default_colors sets the colors of the line plots that are to

    be cycled through. The default correspond to the color scheme of Julia Dynamics.

    • `indicatornames = defaultindicatorlabel(res), chametricnames =

    defaultchametriclabel(res)sets the labels for the indicators and the change metrics, with the default inferring them from the names ofres.config.indicatorsandres.config.change_metrics`.

    • accent_linewidth = 3 sets the line width for the original signals (the

    surrogates have linewidth = 1)

    source
    TransitionsInTimeseries.plot_significance!Function
    plot_significance!(axs, res, signif)

    Update the axs::Matrix{Axis} of a figure obtained with plot_indicator_changes(res) with the signif::SurrogatesSignificance.

    Keyword arguments:

    • flags = nothing provides the significance flags, for instance obtained by

    thresholding the pvalues.

    • nsurro = 20 sets the number of surrogates to plot.
    source
    TransitionsInTimeseries.plot_changes_significanceFunction
    plot_changes_significance(res, signif) → (fig, axs)

    Return fig::Figure and axs::Matrix{Axis}, on which res::ChangesResults and signif::SurrogatesSignificance have been visualised. The source code is as simple as:

    fig, axs = plot_indicator_changes(res; kwargs...)
    +plot_significance!(axs, res, signif; kwargs...)

    For more information, refer to plot_indicator_changes and plot_significance!.

    source

    Utils

    default_window_width
    diff --git a/dev/devdocs/index.html b/dev/devdocs/index.html index a9cef4b..78ed77a 100644 --- a/dev/devdocs/index.html +++ b/dev/devdocs/index.html @@ -1,2 +1,2 @@ -Developer's documentation · TransitionsInTimeseries.jl

    Developer's documentation

    This documentation addresses users that would like to contribute to the software by either solving bugs, improving documentation, or adding new methods. All contributions come in the form of Pull Requests, for which we strongly advise to follow good practices in scientific code, which means that they are properly formatted, documented, tested, etc.

    New indicators or change metrics

    As explained already in e.g., SlidingWindowConfig, new indicators or change metrics are standard Julia functions, so you only need to define such a function (and document it, test it, etc.).

    New pipeline for estimating changes

    This means to contribute a fundamentally new "pipeline" for estimating/detecting transitions in timeseries. This new pipeline defines what a "transition" means. To add a new pipeline follow these steps:

    1. Define a new subtype of ChangesConfig
    2. Define a new subtype of ChangesResults
    3. Add a method for estimate_changes which accepts the new ChangesConfig subtype you defined and returns the ChangesResults subtype you defined.

    And that's it!

    Optionally you can further extend:

    • TransitionsInTimeseries.plot_indicator_changes with the new ChangesResults type you defined. Note the plotting functions are in the ext folder of the repository.
    • significant_transitions(res::ChangesResults, signif::Significance) with your new ChangesResults type and as many Significance subtypes you have the capacity to extend for.

    New pipeline for estimating significance

    Statistically significant changes are "transitions". However, what "significant" means is not universal. There are different ways to test for significance and TransitionsInTimeseries.jl allows various methods.

    To add a new pipeline for estimating significance follow these steps:

    1. Define a new subtype of Significance
    2. Extend significant_transitions(res::ChangesResults, signif::Significance) with your new type and as many ChangesResults subtypes as you have capacity for.

    And that's it! Unfortunately so far we have found no way to make the significant_transitions code agnostic of the changes result, so you would need to add a method manually for every ChangesResults.

    Optionally you can further extend TransitionsInTimeseries.plot_significance! (which you can find in the ext folder) for visualizing the significance results.

    +Developer's documentation · TransitionsInTimeseries.jl

    Developer's documentation

    This documentation addresses users that would like to contribute to the software by either solving bugs, improving documentation, or adding new methods. All contributions come in the form of Pull Requests, for which we strongly advise to follow good practices in scientific code, which means that they are properly formatted, documented, tested, etc.

    New indicators or change metrics

    As explained already in e.g., SlidingWindowConfig, new indicators or change metrics are standard Julia functions, so you only need to define such a function (and document it, test it, etc.).

    New pipeline for estimating changes

    This means to contribute a fundamentally new "pipeline" for estimating/detecting transitions in timeseries. This new pipeline defines what a "transition" means. To add a new pipeline follow these steps:

    1. Define a new subtype of ChangesConfig
    2. Define a new subtype of ChangesResults
    3. Add a method for estimate_changes which accepts the new ChangesConfig subtype you defined and returns the ChangesResults subtype you defined.

    And that's it!

    Optionally you can further extend:

    • TransitionsInTimeseries.plot_indicator_changes with the new ChangesResults type you defined. Note the plotting functions are in the ext folder of the repository.
    • significant_transitions(res::ChangesResults, signif::Significance) with your new ChangesResults type and as many Significance subtypes you have the capacity to extend for.

    New pipeline for estimating significance

    Statistically significant changes are "transitions". However, what "significant" means is not universal. There are different ways to test for significance and TransitionsInTimeseries.jl allows various methods.

    To add a new pipeline for estimating significance follow these steps:

    1. Define a new subtype of Significance
    2. Extend significant_transitions(res::ChangesResults, signif::Significance) with your new type and as many ChangesResults subtypes as you have capacity for.

    And that's it! Unfortunately so far we have found no way to make the significant_transitions code agnostic of the changes result, so you would need to add a method manually for every ChangesResults.

    Optionally you can further extend TransitionsInTimeseries.plot_significance! (which you can find in the ext folder) for visualizing the significance results.

    diff --git a/dev/examples/do-events/index.html b/dev/examples/do-events/index.html index 065af80..3690958 100644 --- a/dev/examples/do-events/index.html +++ b/dev/examples/do-events/index.html @@ -188,4 +188,4 @@ vlines!(axs[1], t_transitions[j], color = Cycled(1), linewidth = 3) push!(figvec, fig) end -figvec[1]Example block output

    It here appears that not all transitions are preceeded by a significant increase of variance and AC1, even in the case of clean and evenly sampled time series. Let's check another case:

    figvec[2]
    Example block output

    Same here! Although CLIMBER-X does not represent real DO-events, the above-performed analysis might be hinting at the fact that not all DO transitions can be forecasted with CSD. Nonetheless, performing a CSD analysis can inform on the evolution of a system's resilience.

    +figvec[1]Example block output

    It here appears that not all transitions are preceeded by a significant increase of variance and AC1, even in the case of clean and evenly sampled time series. Let's check another case:

    figvec[2]
    Example block output

    Same here! Although CLIMBER-X does not represent real DO-events, the above-performed analysis might be hinting at the fact that not all DO transitions can be forecasted with CSD. Nonetheless, performing a CSD analysis can inform on the evolution of a system's resilience.

    diff --git a/dev/examples/ks_paleojump/7f9aca8d.png b/dev/examples/ks_paleojump/7f9aca8d.png deleted file mode 100644 index 912672a..0000000 Binary files a/dev/examples/ks_paleojump/7f9aca8d.png and /dev/null differ diff --git a/dev/examples/ks_paleojump/d3fb78c5.png b/dev/examples/ks_paleojump/d3fb78c5.png new file mode 100644 index 0000000..21d9aa8 Binary files /dev/null and b/dev/examples/ks_paleojump/d3fb78c5.png differ diff --git a/dev/examples/ks_paleojump/index.html b/dev/examples/ks_paleojump/index.html index 99a9e9b..49ce5f1 100644 --- a/dev/examples/ks_paleojump/index.html +++ b/dev/examples/ks_paleojump/index.html @@ -37,7 +37,7 @@ density!(ax, q; label = "D_KS = $(D_KS)") end axislegend(ax) -figExample block output

    Perform the sliding window analysis

    This is just a straightforward call to estimate_changes. In fact, it is even simpler than the tutorial. Here we skip completely the "indicator" estimation step, and we evaluate the change metric directly on input data. We do this by simply passing nothing as the indicators.

    using TransitionsInTimeseries
    +fig
    Example block output

    Perform the sliding window analysis

    This is just a straightforward call to estimate_changes. In fact, it is even simpler than the tutorial. Here we skip completely the "indicator" estimation step, and we evaluate the change metric directly on input data. We do this by simply passing nothing as the indicators.

    using TransitionsInTimeseries
     
     config = SlidingWindowConfig(nothing, normalized_KS_statistic; width_cha = 500)
     
    @@ -65,4 +65,4 @@
     fig = visualize_results(results)
     axDKS = content(fig[2,1])
     vlines!(axDKS, results.t_change[vec(flags)], color = ("red", 0.25))
    -fig
    Example block output

    We could proceed with a lot of preprocessing as in (Bagniewski et al., 2021) but we skip this here for the sake of simplicity.

    +figExample block output

    We could proceed with a lot of preprocessing as in (Bagniewski et al., 2021) but we skip this here for the sake of simplicity.

    diff --git a/dev/examples/logistic/828538a8.png b/dev/examples/logistic/828538a8.png new file mode 100644 index 0000000..c549fb8 Binary files /dev/null and b/dev/examples/logistic/828538a8.png differ diff --git a/dev/examples/logistic/93fce778.png b/dev/examples/logistic/93fce778.png new file mode 100644 index 0000000..fb99110 Binary files /dev/null and b/dev/examples/logistic/93fce778.png differ diff --git a/dev/examples/logistic/ca9a6665.png b/dev/examples/logistic/ca9a6665.png deleted file mode 100644 index 0084902..0000000 Binary files a/dev/examples/logistic/ca9a6665.png and /dev/null differ diff --git a/dev/examples/logistic/dbc91bc3.png b/dev/examples/logistic/dbc91bc3.png deleted file mode 100644 index 8c06e8d..0000000 Binary files a/dev/examples/logistic/dbc91bc3.png and /dev/null differ diff --git a/dev/examples/logistic/index.html b/dev/examples/logistic/index.html index 345709d..5c9b171 100644 --- a/dev/examples/logistic/index.html +++ b/dev/examples/logistic/index.html @@ -112,10 +112,10 @@ surromethod = RandomFourier() overplot_surrogate_significance!(fig, surromethod) -figExample block output

    Different surrogates

    Random Fourier surrogates perserve the power spectrum of the timeseries, but the power spectrum is a property integrated over the whole timeseries. It doesn't contain any information highlighting the local dynamics or information that preserves the local changes of dynamical behavior.

    A surrogate type that does a better job in preserving local sharp changes in the timeseries (and hence provides stricter surrogate-based significance) is for example RelativePartialRandomization.

    A much better alternative is to use block-shuffled surrogates, which preserve the short term local temporal correlation in the timeseries and hence also preserve local short term sharp changes in the dynamic behavior.

    surromethod = RelativePartialRandomization(0.25)
    +fig
    Example block output

    Different surrogates

    Random Fourier surrogates perserve the power spectrum of the timeseries, but the power spectrum is a property integrated over the whole timeseries. It doesn't contain any information highlighting the local dynamics or information that preserves the local changes of dynamical behavior.

    A surrogate type that does a better job in preserving local sharp changes in the timeseries (and hence provides stricter surrogate-based significance) is for example RelativePartialRandomization.

    A much better alternative is to use block-shuffled surrogates, which preserve the short term local temporal correlation in the timeseries and hence also preserve local short term sharp changes in the dynamic behavior.

    surromethod = RelativePartialRandomization(0.25)
     fig = plot_change_metrics()
     overplot_surrogate_significance!(fig, surromethod, "gray")
    -fig
    Example block output

    Our results have improved. In the permutation entropy, we see only two transitions detected as significant, which is correct: only two real dynamical transitions exist in the data. In the other two indicators we also see fewer transitions, but as we have already discussed, no results with the other indicators should be taken into meaningful consideration, as these indicators are simply inappropriate for what we are looking for here.

    Simpler Significance

    Arguably, exactly because we are using the difference_of_means as a change metric, we may want to be less strict and more simple with our tests for significance. Instead of using SurrogatesSignificance we may use the simpler and much faster SigmaSignificance, which simply claims significant time points whenever a change metric exceeds some pre-defined factor of its timeseries standard deviation.

    fig = plot_change_metrics()
    +fig
    Example block output

    Our results have improved. In the permutation entropy, we see only two transitions detected as significant, which is correct: only two real dynamical transitions exist in the data. In the other two indicators we also see fewer transitions, but as we have already discussed, no results with the other indicators should be taken into meaningful consideration, as these indicators are simply inappropriate for what we are looking for here.

    Simpler Significance

    Arguably, exactly because we are using the difference_of_means as a change metric, we may want to be less strict and more simple with our tests for significance. Instead of using SurrogatesSignificance we may use the simpler and much faster SigmaSignificance, which simply claims significant time points whenever a change metric exceeds some pre-defined factor of its timeseries standard deviation.

    fig = plot_change_metrics()
     flags = significant_transitions(results, SigmaSignificance(factor = 5.0))
     
     # Plot the flags
    @@ -125,4 +125,4 @@
         )
     end
     content(fig[1, 1]).title = "significance from std"
    -fig
    Example block output +figExample block output diff --git a/dev/index.html b/dev/index.html index 381eb71..1095aa9 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -TransitionsInTimeseries.jl · TransitionsInTimeseries.jl

    TransitionsInTimeseries.jl

    TransitionsInTimeseries.jl

    TransitionsInTimeseriesModule

    TransitionsInTimeseries.jl

    CI codecov Package Downloads

    TransitionsInTimeseries.jl is a free and open-source software to easily analyse transitions within timeseries in a reproducible, performant, extensible and reliable way. In contrast to other existing software with similar target application, TransitionsInTimeseries.jl defines a generic interface for how to find transitions and how to test for significance. Within this interface, it is easy to expand the software in three orthogonal ways:

    1. Provide the analysis pipelines with new indicators, which can be either self-written or imported from other packages. In particular, the latter offers thousands of metrics that can indicate transitions right out of the box.
    2. Add new analysis pipelines for finding transitions.
    3. Add new ways for significance testing.

    TransitionsInTimeseries is a registered Julia package and can be installed by running:

    ] add TransitionsInTimeseries

    All further information is provided in the documentation, which you can either find online or build locally by running the docs/make.jl file.

    Alternative names for this package could have been: Early Warning Signals / Resilience Indicators / Regime-Shift Identifiers / Change-Point Detectors, or however else you want to call them!

    source
    Star us on GitHub!

    If you have found this package useful, please consider starring it on GitHub. This gives us an accurate lower bound of the (satisfied) user count.

    Content

    Multi-stable systems can display abrupt transitions between two stability regimes. To predict such transitions in real-world systems solely based on data, mathematical tools have been developed in the last decades. Numerous terminologies have been used for them, such as early warning signals, resilience indicators, regime-shift identifiers, change-point detection and transition indicators. TransitionsInTimeseries.jl sticks to the latter terminology and provides an interface that:

    • Allows a fast computation of common transition indicators with a couple of lines, as demonstrated in the example section.
    • Makes the surrogate analysis to test for significance under the hub.
    • Can be easily extended by any user without touching the source code.
    • Reduces the programming overhead for any researcher willing to benchmark new methods.
    • Eases the reproducibility thanks to a clear syntax, a simple installation and RNG-seeded surrogate generation.
    • Increases trustworthiness thanks to a large test suite.
    Similar projects

    An R toolbox and a Python library already exist. However, we believe that they are difficult to extend for the user. Furthermore, they do not offer a native performant code, as here allowed by the use of Julia.

    Approaches

    Over the last decades, research on transition indicators has largely focused on Critical Slowing Down (CSD). CSD is observed when a system approaches a Hopf, a transcritical or a fold bifurcation and consists in a resilience loss of the system. For instance this can be diagnosed by an increase of the variance and the AR1-regression coefficient, as demonstrated in the example section. However, we emphasize that this is one out of many possible approaches for obtaining transition indicators. Recent work has explored new approaches relying on nonlinear dynamics or machine learning. TransitionsInTimeseries.jl is designed to allow these cutting-edge methods and foster the development of new ones.

    +TransitionsInTimeseries.jl · TransitionsInTimeseries.jl

    TransitionsInTimeseries.jl

    TransitionsInTimeseries.jl

    TransitionsInTimeseriesModule

    TransitionsInTimeseries.jl

    CI codecov Package Downloads DOI

    TransitionsInTimeseries.jl is a free and open-source software to easily analyse transitions within timeseries in a reproducible, performant, extensible and reliable way. In contrast to other existing software with similar target application, TransitionsInTimeseries.jl defines a generic interface for how to find transitions and how to test for significance. Within this interface, it is easy to expand the software in three orthogonal ways:

    1. Provide the analysis pipelines with new indicators, which can be either self-written or imported from other packages. In particular, the latter offers thousands of metrics that can indicate transitions right out of the box.
    2. Add new analysis pipelines for finding transitions.
    3. Add new ways for significance testing.

    TransitionsInTimeseries is a registered Julia package and can be installed by running:

    ] add TransitionsInTimeseries

    All further information is provided in the documentation, which you can either find online or build locally by running the docs/make.jl file.

    Alternative names for this package could have been: Early Warning Signals / Resilience Indicators / Regime-Shift Identifiers / Change-Point Detectors, or however else you want to call them!

    source
    Star us on GitHub!

    If you have found this package useful, please consider starring it on GitHub. This gives us an accurate lower bound of the (satisfied) user count.

    Content

    Multi-stable systems can display abrupt transitions between two stability regimes. To predict such transitions in real-world systems solely based on data, mathematical tools have been developed in the last decades. Numerous terminologies have been used for them, such as early warning signals, resilience indicators, regime-shift identifiers, change-point detection and transition indicators. TransitionsInTimeseries.jl sticks to the latter terminology and provides an interface that:

    • Allows a fast computation of common transition indicators with a couple of lines, as demonstrated in the example section.
    • Makes the surrogate analysis to test for significance under the hub.
    • Can be easily extended by any user without touching the source code.
    • Reduces the programming overhead for any researcher willing to benchmark new methods.
    • Eases the reproducibility thanks to a clear syntax, a simple installation and RNG-seeded surrogate generation.
    • Increases trustworthiness thanks to a large test suite.
    Similar projects

    An R toolbox and a Python library already exist. However, we believe that they are difficult to extend for the user. Furthermore, they do not offer a native performant code, as here allowed by the use of Julia.

    Approaches

    Over the last decades, research on transition indicators has largely focused on Critical Slowing Down (CSD). CSD is observed when a system approaches a Hopf, a transcritical or a fold bifurcation and consists in a resilience loss of the system. For instance this can be diagnosed by an increase of the variance and the AR1-regression coefficient, as demonstrated in the example section. However, we emphasize that this is one out of many possible approaches for obtaining transition indicators. Recent work has explored new approaches relying on nonlinear dynamics or machine learning. TransitionsInTimeseries.jl is designed to allow these cutting-edge methods and foster the development of new ones.

    diff --git a/dev/refs/index.html b/dev/refs/index.html index d71a1b4..0bc7462 100644 --- a/dev/refs/index.html +++ b/dev/refs/index.html @@ -1,2 +1,2 @@ -References · TransitionsInTimeseries.jl

    References

    +References · TransitionsInTimeseries.jl

    References

    diff --git a/dev/search_index.js b/dev/search_index.js index f9c3908..81d6f04 100644 --- a/dev/search_index.js +++ b/dev/search_index.js @@ -1,3 +1,3 @@ var documenterSearchIndex = {"docs": -[{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"EditURL = \"tutorial.jl\"","category":"page"},{"location":"tutorial/#Tutorial","page":"Tutorial","title":"Tutorial","text":"","category":"section"},{"location":"tutorial/#workflow","page":"Tutorial","title":"Workflow","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Computing transition indicators consists of the following steps:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Doing any preprocessing of raw data first, such as detrending. This not part of TransitionsInTimeseries.jl and yields the input timeseries.\nEstimating the timeseries of an indicator by sliding a window over the input timeseries.\nComputing the changes of the indicator by sliding a window over its timeseries. Alternatively, the change metric can be estimated over the whole segment, as examplified in the section Segmented windows.\nGenerating many surrogates that preserve important statistical properties of the original timeseries.\nPerforming step 2 and 3 for the surrogate timeseries.\nChecking whether the indicator change timeseries of the real timeseries shows a significant feature (trend, jump or anything else) when compared to the surrogate data.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"These steps are illustrated one by one in the tutorial below, and then summarized in the convenient API that TransitionsInTimeseries.jl exports.","category":"page"},{"location":"tutorial/#example_stepbystep","page":"Tutorial","title":"Tutorial – Educational","text":"","category":"section"},{"location":"tutorial/#Raw-input-data","page":"Tutorial","title":"Raw input data","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Let us load data from a bistable nonlinear model subject to noise and to a gradual change of the forcing that leads to a transition. Furthermore, we also load data from a linear model, which is by definition monostable and therefore incapable of transitioning. This is done to control the rate of false positives, a common problem that can emerge when looking for transition indicators. The models are governed by:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"dfracmathrmdx_lmathrmdt = - x_l - 1 + f(t) + n(t) \ndfracmathrmdx_nlmathrmdt = - x_nl^3 + x_nl + f(t) + n(t)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"with x_l the state of the linear model, x_nl the state of the bistable model, f the forcing and n the noise. For f=0 they both display an equilibrium point at x=-1. However, the bistable model also displays a further equilibrium point at x=1. Loading (and visualizing with Makie) such prototypical data to test some indicators can be done by simply running:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using TransitionsInTimeseries, CairoMakie\n\nt, x_linear, x_nlinear = load_linear_vs_doublewell()\nfig, ax = lines(t, x_linear)\nlines!(ax, t, x_nlinear)\nax.title = \"raw data\"\nfig","category":"page"},{"location":"tutorial/#Preprocessing","page":"Tutorial","title":"Preprocessing","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"note: Not part of TransitionsInTimeseries.jl\nAny timeseries preprocessing, such as the de-trending step we do here, is not part of TransitionsInTimeseries.jl and is the responsibility of the researcher.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The nonlinear system clearly displays a transition between two stability regimes. To forecast such transition, we analyze the fluctuations of the timeseries around the attractor, assumed to be tracked. Therefore, a detrending step is needed - here simply obtained by building the difference of the timeseries with lag 1.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"x_l_fluct = diff(x_linear)\nx_nl_fluct = diff(x_nlinear)\ntfluct = t[2:end]\n\nfig, ax = lines(tfluct, x_l_fluct)\nlines!(ax, tfluct, x_nl_fluct .+ 0.05)\nax.title = \"input timeseries\"\nfig","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"At this point, x_l_fluct and x_nl_fluct are considered the input timeseries.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"info: Detrending in Julia\nDetrending can be performed in many ways. A wide range of Julia packages exists to perform smoothing such as Loess.jl or DSP.jl. There the detrending step consists of subtracting the smoothed signal from the original one.","category":"page"},{"location":"tutorial/#Indicator-timeseries","page":"Tutorial","title":"Indicator timeseries","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"We can then compute the values of some \"indicator\" (a Julia function that inputs a timeseries and outputs a number). An indicator should be a quantity that is likely to change if a transition occurs, or is about to occur in the timeseries. We compute indicators by applying a sliding window over the input timeseries, determined by the width and the stride with which it is applied. Here we demonstrate this computation with the AR1-regression coefficient (under white-noise assumption), implemented as ar1_whitenoise:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"indicator = ar1_whitenoise\nindicator_window = (width = 400, stride = 1)\n\n# By mapping `last::Function` over a windowviewer of the time vector,\n# we obtain the last time step of each window.\n# This therefore only uses information from `k-width+1` to `k` at time step `k`.\n# Alternatives: `first::Function`, `midpoint:::Function`.\nt_indicator = windowmap(last, tfluct; indicator_window...)\nindicator_l = windowmap(indicator, x_l_fluct; indicator_window...)\nindicator_nl = windowmap(indicator, x_nl_fluct; indicator_window...)\n\nfig, ax = lines(t_indicator, indicator_l)\nlines!(ax, t_indicator, indicator_nl)\nax.title = \"indicator timeseries\"\nfig","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The lines plotted above are the indicator timeseries.","category":"page"},{"location":"tutorial/#Change-metric-timeseries","page":"Tutorial","title":"Change metric timeseries","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"From here, we process the indicator timeseries to quantify changes in it. This step is in essence the same as before: we apply some function over a sliding window of the indicator timeseries. We call this new timeseries the change metric timeseries. In the example here, the change metric we will employ will be the slope (over a sliding window), calculated via means of a RidgeRegressionSlope:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"change_window = (width = 30, stride = 1)\nridgereg = RidgeRegressionSlope(lambda = 0.0)\nprecompridgereg = precompute(ridgereg, t[1:change_window.width])\n\nt_change = windowmap(last, t_indicator; change_window...)\nchange_l = windowmap(precompridgereg, indicator_l; change_window...)\nchange_nl = windowmap(precompridgereg, indicator_nl; change_window...)\n\nfig, ax = lines(t_change, change_l)\nlines!(ax, t_change, change_nl)\nax.title = \"change metric timeseries\"\nfig","category":"page"},{"location":"tutorial/#Timeseries-surrogates","page":"Tutorial","title":"Timeseries surrogates","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"As expected from Critical Slowing Down, an increase of the AR1-regression coefficient can be observed. Although eyeballing the timeseries might already be suggestive, we want a rigorous framework for testing for significance.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"In TransitionsIdentifiers.jl we perform significance testing using the method of timeseries surrogates and the TimeseriesSurrogates.jl Julia package. This has the added benefits of reproducibility, automation and flexibility in choosing the surrogate generation method. Note that TimeseriesSurrogates is re-exported by TransitionsInTimeseries, so that you don't have to using both of them.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"To illustrate the surrogate, we compare the change metric computed from the bistable timeseries what that computed from a surrogate of the same timeseries.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"# Generate Fourier random-phase surrogates\nusing Random: Xoshiro\ns = surrogate(x_nl_fluct, RandomFourier(), Xoshiro(123))\n\nfunction gridfig(nrows, ncols)\n fig = Figure()\n axs = [Axis(fig[i, j], xticklabelsvisible = i == nrows ? true : false)\n for j in 1:ncols, i in 1:nrows]\n rowgap!(fig.layout, 10)\n return fig, axs\nend\nfig, axs = gridfig(2, 1)\nlines!(axs[1], tfluct, x_nl_fluct, color = Cycled(2))\nlines!(axs[1], tfluct, s .- 0.05, color = Cycled(3))\naxs[1].title = \"real signal vs. surrogate(s)\"\n\n# compute and plot indicator and change metric\nindicator_s = windowmap(indicator, s; indicator_window...)\nchange_s = windowmap(precompridgereg, indicator_s; change_window...)\n\nlines!(axs[2], t_change, change_nl, label = \"nonlin\", color = Cycled(2))\nlines!(axs[2], t_change, change_s, label = \"surrogate\", color = Cycled(3))\naxislegend()\naxs[2].title = \"change metric\"\n\n[xlims!(ax, 0, 50) for ax in axs]\nfig","category":"page"},{"location":"tutorial/#Quantifying-significance","page":"Tutorial","title":"Quantifying significance","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"To quantify the significance of the values of the change metric timeseries we perform a standard surrogate test by computing the p-value w.r.t. the change metrics of thousands of surrogates of the input timeseries. A low p-value (typically p<0.05) is commonly considered as significant. To visualize significant trends, we plot the p-value vs. time:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"n_surrogates = 1_000\nfig, axs = gridfig(2, 2)\naxs[1].title = \"linear\"\naxs[2].title = \"nonlinear\"\n\nfor (j, ax, axsig, x) in zip(1:2, axs[1:2], axs[3:4], (x_l_fluct, x_nl_fluct))\n\n orig_change = j == 1 ? change_l : change_nl\n sgen = surrogenerator(x, RandomFourier(), Xoshiro(123))\n pval = zeros(length(change_s))\n\n # Collect all surrogate change metrics\n for i in 1:n_surrogates\n s = sgen()\n indicator_s = windowmap(indicator, s; indicator_window...)\n change_s = windowmap(precompridgereg, indicator_s; change_window...)\n pval += orig_change .< change_s\n end\n\n pval ./= n_surrogates\n lines!(ax, t_change, orig_change) # ; color = Cycled(j)\n lines!(axsig, t_change, pval) # ; color = Cycled(j+2)\nend\n\n[xlims!(ax, 0, 50) for ax in axs]\nfig","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"As expected, the data generated by the nonlinear model displays a significant increase of the AR1-regression coefficient before the transition, which is manifested by a low p-value. In contrast, the data generated by the linear model does not show anything similar.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Performing the step-by-step analysis of transition indicators is possible and might be preferred for users wanting high flexibility. However, this results in a substantial amount of code. We therefore provide convenience functions that wrap this analysis, as shown in the next section.","category":"page"},{"location":"tutorial/#example_fastforward","page":"Tutorial","title":"Tutorial – TransitionsInTimeseries.jl","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"TransitionsInTimeseries.jl wraps this typical workflow into a simple, extendable, and modular API that researchers can use with little effort. In addition, it allows performing the same analysis for several indicators / change metrics in one go.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The interface is simple, and directly parallelizes the Workflow. It is based on the creation of a ChangesConfig, which contains a list of indicators, and corresponding metrics, to use for doing the above analysis. It also specifies what kind of surrogates to generate.","category":"page"},{"location":"tutorial/#Sliding-windows","page":"Tutorial","title":"Sliding windows","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The following blocks illustrate how the above extensive example is re-created in TransitionsInTimeseries.jl","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using TransitionsInTimeseries, CairoMakie\n\nt, x_linear, x_nlinear = load_linear_vs_doublewell()\n\n# input timeseries and time\ninput = x_nl_fluct = diff(x_nlinear)\nt = t[2:end]\n\nfig, ax = lines(t, input)\nax.title = \"input timeseries\"\nfig","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"To perform all of the above analysis we follow a 2-step process.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Step 1, we decide what indicators and change metrics to use in SlidingWindowConfig and apply those via a sliding window to the input timeseries using estimate_changes.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"# These indicators are suitable for Critical Slowing Down\nindicators = (var, ar1_whitenoise)\n\n# use the ridge regression slope for both indicators\nchange_metrics = (RidgeRegressionSlope(), RidgeRegressionSlope())\n\n# choices go into a configuration struct\nconfig = SlidingWindowConfig(indicators, change_metrics;\n width_ind = 400, width_cha = 30, whichtime = last)\n\n# choices are processed\nresults = estimate_changes(config, input, t)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"We can conveniently plot the information contained in results by using plot_indicator_changes:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"fig = plot_indicator_changes(results)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Step 2 is to estimate significance using SurrogatesSignificance and the function significant_transitions. Finally, we can conveniently plot the results obtained by updating the figure obtained above with plot_significance!:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"signif = SurrogatesSignificance(n = 1000, tail = [:right, :right])\nflags = significant_transitions(results, signif)\nplot_significance!(fig, results, signif, flags = flags)\nfig","category":"page"},{"location":"tutorial/#segmented_windows","page":"Tutorial","title":"Segmented windows","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The analysis shown so far relies on sliding windows of the change metric. This is particularly convenient for transition detection tasks. Segmented windows for the change metric computation are however preferable when it comes to prediction tasks. By only slightly modifying the syntax used so far, one can perform the same computations on segmented windows, as well as visualise the results conveniently:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"config = SegmentedWindowConfig(indicators, change_metrics,\n t[1:1], t[1200:1200]; whichtime = last, width_ind = 200,\n min_width_cha = 100)\nresults = estimate_changes(config, input, t)\nsignif = SurrogatesSignificance(n = 1000, tail = [:right, :right])\nflags = significant_transitions(results, signif)\nfig = plot_changes_significance(results, signif)","category":"page"},{"location":"api/#API","page":"API","title":"API","text":"","category":"section"},{"location":"api/#Main-analysis-functions","page":"API","title":"Main analysis functions","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"estimate_changes\nChangesConfig\nChangesResults","category":"page"},{"location":"api/#TransitionsInTimeseries.estimate_changes","page":"API","title":"TransitionsInTimeseries.estimate_changes","text":"estimate_changes(config::ChangesConfig, x [,t]) → result\n\nEstimate possible transitions for input timeseries x using the approach specified in the configuration type config, see ChangesConfig for possibilities. t is the time vector corresponding to x, which defaults to eachindex(x).\n\nReturn the output as subtype of ChangesResults. The particular form of the output depends on the config and is described in its docstring. Regardless of type, result can always be given to significant_transitions to deduce which possible transitions are statistically significant using a variety of significance tests.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.ChangesConfig","page":"API","title":"TransitionsInTimeseries.ChangesConfig","text":"ChangesConfig\n\nSupertype for how \"changes\" in a timeseries are estimated in estimate_changes. \"Changes\" deemed statistically significant in significant_transitions are \"transitions\" in the timeseries.\n\nExisting subtypes of ChangesConfig are:\n\nSlidingWindowConfig.\nSegmentedWindowConfig.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.ChangesResults","page":"API","title":"TransitionsInTimeseries.ChangesResults","text":"ChangesResults\n\nSupertype used to gather results of estimate_changes. The concrete subtype instances are described in the docstrings of configuration types.\n\n\n\n\n\n","category":"type"},{"location":"api/#Sliding-window","page":"API","title":"Sliding window","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"SlidingWindowConfig\nSlidingWindowResults","category":"page"},{"location":"api/#TransitionsInTimeseries.SlidingWindowConfig","page":"API","title":"TransitionsInTimeseries.SlidingWindowConfig","text":"SlidingWindowConfig <: ChangesConfig\nSlidingWindowConfig(indicators, change_metrics; kwargs...)\n\nA configuration that can be given to estimate_changes. It estimates transitions by a sliding window approach:\n\nEstimate the timeseries of an indicator by sliding a window over the input timeseries.\nEstimate changes of an indicator by sliding a window of the change metric over the indicator timeseries.\n\nBoth indicators and change metrics are generic Julia functions that input an x::AbstractVector and output an s::Real. Any function may be given and see making custom indicators/change metrics in the documentation for more information on possible optimizations.\n\nindicators can be a single function or a tuple of indicators. Similarly, change_metrics can be a tuple or a single function. If tuples, the length of indicators and change_metrics must match. This way the analysis can be efficiently repeated for many indicators and/or change metrics.\n\nThe results output corresponding to SlidingWindowConfig is SlidingWindowResults.\n\nStep 1. is skipped if nothing is provided as indicators, in which case the change metrics are estimated directly from input data.\n\nKeyword arguments\n\nwidth_ind::Int=100, stride_ind::Int=1: width and stride given to WindowViewer to compute the indicator from the input timeseries.\nwidth_cha::Int=50, stride_cha::Int=1: width and stride given to WindowViewer to compute the change metric timeseries from the indicator timeseries.\nwhichtime = midpoint: The time vector corresponding to the indicators / change metric timeseries is obtained from t in estimate_changes using the keyword whichtime. Options include:\nlast: use the last timepoint of each window\nmidpoint: use the mid timepoint of each time window\nfirst: use first timepoint of each window\nIn fact, the indicators time vector is computed simply via\nt_indicator = windowmap(whichtime, t; width_ind, stride_ind)\nt_change = windowmap(whichtime, t_indicator; width_cha, stride_cha)\nso any other function of the time window may be given to extract the time point itself, such as mean or median.\nT = Float64: Element type of input timeseries to initialize some computations.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SlidingWindowResults","page":"API","title":"TransitionsInTimeseries.SlidingWindowResults","text":"SlidingWindowResults <: ChangesResults\n\nA struct containing the output of estimate_changes used with SlidingWindowConfig. It can be used for further analysis, visualization, or given to significant_transitions.\n\nIt has the following fields that the user may access\n\nx: the input timeseries.\nt: the time vector of the input timeseries.\nx_indicator, the indicator timeseries (matrix with each column one indicator).\nt_indicator, the time vector of the indicator timeseries.\nx_change, the change metric timeseries (matrix with each column one change metric).\nt_change, the time vector of the change metric timeseries.\nconfig::SlidingWindowConfig, what was used for the analysis.\n\n\n\n\n\n","category":"type"},{"location":"api/#Segmented-window","page":"API","title":"Segmented window","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"SegmentedWindowConfig\nSegmentedWindowResults","category":"page"},{"location":"api/#TransitionsInTimeseries.SegmentedWindowConfig","page":"API","title":"TransitionsInTimeseries.SegmentedWindowConfig","text":"SegmentedWindowConfig <: IndicatorsChangeConfig\nSegmentedWindowConfig(indicators, change_metrics, tseg_start, tseg_end; kwargs...)\n\nA configuration that can be given to estimate_changes. It estimates transitions by estimating indicators and changes in user-defined window segments as follows:\n\nFor each segment specified, estimate the corresponding indicator timeseries by sliding a window over the input timeseries (within the window segment).\nFor each segment of the indicator timeseries, estimate a scalar change metric by applying the change metric over the full segment of the indicator timeseries.d\n\ntseg_start, tseg_end are the starts and ends of the window segments (the window segments may overlap, that's okay). indicators, change_metrics are identical as in SlidingWindowConfig.\n\nThe results output corresponding to SlidingWindowConfig is SegmentedWindowResults.\n\nKeyword arguments\n\nwidth_ind::Int=100, stride_ind::Int=1, whichtime = midpoint, T = Float64: keywords identical as in SlidingWindowConfig.\nmin_width_cha::Int=typemax(Int): minimal width required to perform the change metric estimation. If a segment is not sufficiently long, the change metric is NaN.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SegmentedWindowResults","page":"API","title":"TransitionsInTimeseries.SegmentedWindowResults","text":"SegmentedWindowResults <: ChangesResults\n\nA struct containing the output of estimate_changes used with SegmentedWindowConfig. It can be used for further analysis, visualization, or given to significant_transitions.\n\nIt has the following fields that the user may access\n\nx: the input timeseries.\nt: the time vector of the input timeseries.\nx_indicator::Vector{Matrix}, with x_indicator[k] the indicator timeseries (matrix with each column one indicator) of the k-th segment.\nt_indicator::Vector{Vector}, with t_indicator[k] the time vector of the indicator timeseries for the k-th segment.\nx_change::Matrix, the change metric values with x[k, i] the change metric of the i-th indicator for the k-th segment.\nt_change, the time vector of the change metric.\nconfig::SegmentedWindowConfig, what was used for the analysis.\ni1::Vector{Int} indices corresponding to start time of each segment.\ni2::Vector{Int} indices corresponding to end time of each segment.\nprecomp_change_metrics vector containing the precomputed change metrics of each segment.\n\n\n\n\n\n","category":"type"},{"location":"api/#Slope-change","page":"API","title":"Slope change","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"SlopeChangeConfig\nSlopeChangeResults","category":"page"},{"location":"api/#TransitionsInTimeseries.SlopeChangeConfig","page":"API","title":"TransitionsInTimeseries.SlopeChangeConfig","text":"SlopeChangeConfig <: ChangesConfig\nSlopeChangeConfig(; indicator = nothing, kw...)\n\nA configuration that can be given to estimate_changes. It estimates a change of slope in the timeseries by fitting two connected linear segments to the timeseries, returning the results (i.e., the two-linear fits) as SlopeChangeResults.\n\nKeyword arguments\n\nindicator = nothing: if not nothing. Otherwise it should be a function f(x) -> Real. The slope fitting is then done over an indicator of the timeseries, which itself is estimated via a sliding window exactly as in SlidingWindowConfig.\nwidth_ind, stride_ind, whichtime: exactly as in SlidingWindowConfig if indicator is not nothing.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SlopeChangeResults","page":"API","title":"TransitionsInTimeseries.SlopeChangeResults","text":"SlopeChangeResults <: ChangesResults\n\nA struct containing the output of estimate_changes used with SlopeChangeConfig. It can be used for further analysis, visualization, or given to significant_transitions. The only significance type that you can use this with significant_transitions is SlopeChangeSignificance.\n\nIt has the following fields that the user may access:\n\nx: the input timeseries.\nt: the time vector of the input timeseries.\nx_indicator, the indicator timeseries.\nt_indicator, the time vector of the indicator timeseries.\nt_change, the time the slope changes.\nfitparams = a, b, c, d, the fitted linear coefficients, a + b*t before. t_change and c + d*t after t_change.\n\n\n\n\n\n","category":"type"},{"location":"api/#Significance-testing","page":"API","title":"Significance testing","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"significant_transitions\nSignificance\nSurrogatesSignificance\nThresholdSignificance\nSigmaSignificance\nQuantileSignificance\nSlopeChangeSignificance","category":"page"},{"location":"api/#TransitionsInTimeseries.significant_transitions","page":"API","title":"TransitionsInTimeseries.significant_transitions","text":"significant_transitions(res::ChangesResults, signif::Significance)\n\nEstimate significant transtions in res using the method described by signif. Return flags, a Boolean matrix with identical size as the changes stored in res (which typically is stored in the field res.x_change). flags is true wherever a change metric of res is deemed significant.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.Significance","page":"API","title":"TransitionsInTimeseries.Significance","text":"Significance\n\nSupertype used to test for significance in significant_transitions. Changes that are statistically significant are \"transitions\".\n\nValid subtypes are:\n\nSurrogatesSignificance.\nSigmaSignificance.\nQuantileSignificance.\nThresholdSignificance.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SurrogatesSignificance","page":"API","title":"TransitionsInTimeseries.SurrogatesSignificance","text":"SurrogatesSignificance <: Significance\nSurrogatesSignificance(; kwargs...)\n\nA configuration struct for significance testing significant_transitions using timeseries surrogates.\n\nKeyword arguments\n\nsurromethod = RandomFourier(): method to generate surrogates\nn = 1000: how many surrogates to generate\nrng = Random.default_rng(): random number generator for the surrogates\np = 0.05: threshold for significance of the p-value\ntail = :both: tail type used, see below\n\nDescription\n\nWhen used with ChangesResults, significance is estimated as follows: n surrogates from the input timeseries are generated using surromethod, which is any Surrogate subtype provided by TimeseriesSurrogates.jl. For each surrogate, the indicator and then change metric timeseries is extracted. The values of the surrogate change metrics form a distribution of values (one at each time point). The value of the original change metric is compared to that of the surrogate distribution and a p-value is extracted according to the specified tail. The p-value is compared with p to claim significance. After using SurrogatesSignificance, you may access the full p-values before thresholding in the field .pvalues (to e.g., threshold with different p).\n\nThe p-value is simply the proportion of surrogate change metric values that exceed (for tail = :right) or subseed (tail = :left) the original change metric at each given time point. Use tail = :left if the surrogate data are expected to have higher change metric values. This is the case for statistics that quantify entropy. For statistics that quantify autocorrelation, use tail = :right instead. For anything else, use the default tail = :both. An iterable of tail values can also be given, in which case a specific tail is used for each change metric in ChangesResults.\n\nNote that the raw p-values can be accessed in the field .pvalues, after calling the significant_transitions function with SurrogatesSignificance, in case you wish to obtain a different threshold of the p-values.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.ThresholdSignificance","page":"API","title":"TransitionsInTimeseries.ThresholdSignificance","text":"ThresholdSignificance(threshold::Real; tail = :right) <: Significance\n\nA configuration struct for significance testing in significant_transitions. Significance is estimated by comparing the value of each change metric with the given threshold. Values that exceed the threshold (if tail = :right) or subseed the threshold (if tail = :left) are deemed significant. If tail = :both then either condition is checked.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SigmaSignificance","page":"API","title":"TransitionsInTimeseries.SigmaSignificance","text":"SigmaSignificance(; factor = 3.0, tail = :both) <: Significance\n\nA configuration struct for significance testing significant_transitions. When used with ChangesResults, significance is estimated by comparing how many standard deviations (σ) the value exceeds the mean value (μ). Values that exceed (if tail = :right) μ + factor*σ, or subseed (if tail = :left) μ - factor*σ are deemed significant. If tail = :both then either condition is checked.\n\nfactor can also be a vector of values, in which case a different value is used for each change metric.\n\nSee also QuantileSignificance.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.QuantileSignificance","page":"API","title":"TransitionsInTimeseries.QuantileSignificance","text":"QuantileSignificance(; p = 0.95, tail = :both) <: Significance\n\nA configuration struct for significance testing significant_transitions. When used with ChangesResults, significance is estimated by comparing the value of each change metric with its p-quantile. Values that exceed the p-quantile (if tail = :right) or subseed the 1-p-quantile (if tail = :left) are deemed significant. If tail = :both then either condition is checked.\n\nQuantileSignficance guarantees that some values will be significant by the very definition of what a quantile is. See also SigmaSignificance that is similar but does not have this guarantee.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SlopeChangeSignificance","page":"API","title":"TransitionsInTimeseries.SlopeChangeSignificance","text":"SlopeChangeSignificance(; moe_slope, moe_offset, slope_diff = moe_slope, pvalue = 0.05)\n\nTest whether the result of SlopeChangeResults is statistically significant.\n\nTwo tests are done:\n\nCheck whether the margin of error of the fitted parameters a, b, c, d of the two linear segments a + b*t, c + d*t is less than the specified margins of error, for a chosen pvalue.\nTest that the two slopes b, d have difference greater than slope_diff.\n\nThe Boolean & of the above two is the final test.\n\nThe margin of error is simply half the size of the confidence interval, also known as radius of the confidence interval.\n\n\n\n\n\n","category":"type"},{"location":"api/#indicators","page":"API","title":"Indicators","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Note that any Julia function can be an indicator or change metric, so the list here is only just a couple of indicators directly implemented in this package.","category":"page"},{"location":"api/#Value-distribution","page":"API","title":"Value distribution","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"StatsBase.mean\nStatsBase.skewness\nStatsBase.kurtosis","category":"page"},{"location":"api/#Statistics.mean","page":"API","title":"Statistics.mean","text":"mean(A::AbstractArray, w::AbstractWeights[, dims::Int])\n\nCompute the weighted mean of array A with weight vector w (of type AbstractWeights). If dim is provided, compute the weighted mean along dimension dims.\n\nExamples\n\nn = 20\nx = rand(n)\nw = rand(n)\nmean(x, weights(w))\n\n\n\n\n\n","category":"function"},{"location":"api/#StatsBase.skewness","page":"API","title":"StatsBase.skewness","text":"skewness(v, [wv::AbstractWeights], m=mean(v))\n\nCompute the standardized skewness of a real-valued array v, optionally specifying a weighting vector wv and a center m.\n\n\n\n\n\n","category":"function"},{"location":"api/#StatsBase.kurtosis","page":"API","title":"StatsBase.kurtosis","text":"kurtosis(v, [wv::AbstractWeights], m=mean(v))\n\nCompute the excess kurtosis of a real-valued array v, optionally specifying a weighting vector wv and a center m.\n\n\n\n\n\n","category":"function"},{"location":"api/#Critical-Slowing-Down","page":"API","title":"Critical Slowing Down","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"StatsBase.var\nar1_whitenoise","category":"page"},{"location":"api/#Statistics.var","page":"API","title":"Statistics.var","text":"var(x::AbstractArray, w::AbstractWeights, [dim]; mean=nothing, corrected=false)\n\nCompute the variance of a real-valued array x, optionally over a dimension dim. Observations in x are weighted using weight vector w. The uncorrected (when corrected=false) sample variance is defined as:\n\nfrac1sumw sum_i=1^n w_ileft(x_i - μright)^2 \n\nwhere n is the length of the input and μ is the mean. The unbiased estimate (when corrected=true) of the population variance is computed by replacing frac1sumw with a factor dependent on the type of weights used:\n\nAnalyticWeights: frac1sum w - sum w^2 sum w\nFrequencyWeights: frac1sumw - 1\nProbabilityWeights: fracn(n - 1) sum w where n equals count(!iszero, w)\nWeights: ArgumentError (bias correction not supported)\n\n\n\n\n\nvar(ce::CovarianceEstimator, x::AbstractVector; mean=nothing)\n\nCompute the variance of the vector x using the estimator ce.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.ar1_whitenoise","page":"API","title":"TransitionsInTimeseries.ar1_whitenoise","text":"ar1_whitenoise(x::AbstractVector)\n\nReturn the AR1 regression coefficient of a time series x by computing the analytic solution of the least-square parameter estimation under white-noise assumption for the data-generating process:\n\ntheta = sum_i=2^n x_i x_i-1 sum_i=2^n x_i-1^2\n\n\n\n\n\n","category":"function"},{"location":"api/#Spectrum","page":"API","title":"Spectrum","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"LowfreqPowerSpectrum\nPrecomputedLowfreqPowerSpectrum","category":"page"},{"location":"api/#TransitionsInTimeseries.LowfreqPowerSpectrum","page":"API","title":"TransitionsInTimeseries.LowfreqPowerSpectrum","text":"LowfreqPowerSpectrum(; q_lofreq = 0.1)\n\nReturn a PrecomputableFunction containing all the necessary fields to generate a PrecomputedLowfreqPowerSpectrum. The latter can be initialized by precompute:\n\nlfps = precompute( LowfreqPowerSpectrum() )\n\nKeyword arguments:\n\nq_lofreq: a number between 0 and 1 that characterises which portion of the\n\nfrequency spectrum is considered to be low. For instance, q_lofreq = 0.1 implies that the lowest 10% of frequencies are considered to be the low ones.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.PrecomputedLowfreqPowerSpectrum","page":"API","title":"TransitionsInTimeseries.PrecomputedLowfreqPowerSpectrum","text":"PrecomputedLowfreqPowerSpectrum\n\nA struct containing all the precomputed fields to efficiently perform repetitive computation of the low-frequency power spectrum (LFPS), a number between 0 and 1 that characterizes the amount of power contained in the low frequencies of the power density spectrum of x. Once lfps::PrecomputedLowfreqPowerSpectrum is initialized, it can be used as a function to obtain the LFPS of x::AbstractVector by:\n\nlfps(x)\n\n\n\n\n\n","category":"type"},{"location":"api/#Nonlinear-dynamics","page":"API","title":"Nonlinear dynamics","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Indicators that come from nonlinear timeseries analysis typically quantify some entropy-like or complexity measure from the timeseries. Thousands (literally) such measures are provided out of the box by the ComplexityMeasures.jl package. Given that any of these may be used as an indicator or change metric, we made the decision to not copy-paste any measure here, as it is easy for the user to use any of them.","category":"page"},{"location":"api/","page":"API","title":"API","text":"For example, using the permutation entropy as an indicator is as simple as doing","category":"page"},{"location":"api/","page":"API","title":"API","text":"using ComplexityMeasures\nest = OrdinalPatterns(; m = 3) # order 3\n# create a function that given timeseries returns permutation entropy\nindicator = x -> entropy_normalized(est, x)","category":"page"},{"location":"api/","page":"API","title":"API","text":"and giving the created indicator to e.g., SlidingWindowConfig.","category":"page"},{"location":"api/#change_metrics","page":"API","title":"Change metrics","text":"","category":"section"},{"location":"api/#Slope","page":"API","title":"Slope","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"kendalltau\nspearman\nRidgeRegressionSlope\nPrecomputedRidgeRegressionSlope","category":"page"},{"location":"api/#TransitionsInTimeseries.kendalltau","page":"API","title":"TransitionsInTimeseries.kendalltau","text":"kendalltau(x::AbstractVector)\n\nCompute the kendall-τ correlation coefficient of the time series x. kendalltau can be used as a change metric focused on trend.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.spearman","page":"API","title":"TransitionsInTimeseries.spearman","text":"spearman(x::AbstractVector)\n\nCompute the spearman correlation coefficient of the time series x. spearman can be used as a change metric focused on trend.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.RidgeRegressionSlope","page":"API","title":"TransitionsInTimeseries.RidgeRegressionSlope","text":"RidgeRegressionSlope(; lambda = 0.0) → rr\n\nReturn a PrecomputableFunction containing all the necessary fields to generate a PrecomputedRidgeRegressionSlope. rr can be used as a change metric focused on trend.\n\nlambda is a regularization constant, usually between 0 and 1.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.PrecomputedRidgeRegressionSlope","page":"API","title":"TransitionsInTimeseries.PrecomputedRidgeRegressionSlope","text":"PrecomputedRidgeRegressionSlope\n\nA struct containing the precomputed ridge regression matrix. Once rrslope::PrecomputedRidgeRegressionSlope is initialized, it can be used as a function to obtain the ridge regression slope of x::AbstractVector by applying:\n\nrrslope(x)\n\n\n\n\n\n","category":"type"},{"location":"api/#Value-distribution-differences","page":"API","title":"Value distribution differences","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"difference_of_means","category":"page"},{"location":"api/#TransitionsInTimeseries.difference_of_means","page":"API","title":"TransitionsInTimeseries.difference_of_means","text":"difference_of_means(x::AbstractArray)\n\nReturn the absolute difference of the means of the first and second halfs of x. difference_of_means can be used as a change metric focused on value differences. Creating similar statistical differences using other moments instead of mean is trivial. In fact, the source of difference_of_means is just:\n\n# assumes 1-based indexing\nn = length(x)\nx1 = view(x, 1:n÷2)\nx2 = view(x, (n÷2 + 1):n)\nreturn abs(mean(x1) - mean(x2))\n\ndifference_of_means can also sensibly be used for windows of size 2, in which case the change metric timeseries is the same as the abs.(diff(...)) of the indicator timeseries.\n\n\n\n\n\n","category":"function"},{"location":"api/#own_indicator","page":"API","title":"Make your own indicator/metric!","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"The only difference between what is an \"indicator\" and what is a \"change metric\" is purely conceptual. As far as the code base of TransitionsInTimeseries.jl is concerned, they are both functions f: x::AbstractVector{Real} -> f(x)::Real. As a user you may give any such function for an indicator or change metric.","category":"page"},{"location":"api/","page":"API","title":"API","text":"There are situations where you may optimize such a function based on knowledge of input x type and length, in which case you may use PrecomputableFunction:","category":"page"},{"location":"api/","page":"API","title":"API","text":"PrecomputableFunction\nprecompute","category":"page"},{"location":"api/#TransitionsInTimeseries.PrecomputableFunction","page":"API","title":"TransitionsInTimeseries.PrecomputableFunction","text":"PrecomputableFunction\n\nSupertype of structs containing the necessary field to precompute a ::Function by:\n\nprecompute(f::PrecomputableFunction, t)\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.precompute","page":"API","title":"TransitionsInTimeseries.precompute","text":"precompute(f, t)\n\nPrecompute the function f given a time vector t.\n\n\n\n\n\n","category":"function"},{"location":"api/#Surrogates","page":"API","title":"Surrogates","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"For the surrogate generation, you can use any subtype of Surrogate defined in Timeseriessurrogates.jl.","category":"page"},{"location":"api/#Sliding-windows","page":"API","title":"Sliding windows","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"WindowViewer\nwindowmap\nwindowmap!","category":"page"},{"location":"api/#TransitionsInTimeseries.WindowViewer","page":"API","title":"TransitionsInTimeseries.WindowViewer","text":"WindowViewer(x; width, stride)\n\nInitialize an iterator that generates views over the given timeseries x based on a window with a given width, incrementing the window views with the given stride. You can use this directly with map, such as map(std, WindowViewer(x, ...)) would give you the moving-window-timeseries of the std of x.\n\nIf not given, the keywords width, stride are respectively taken as default_window_width(x) and 1.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.windowmap","page":"API","title":"TransitionsInTimeseries.windowmap","text":"windowmap(f::Function, x::AbstractVector; kwargs...) → mapped_f\n\nA shortcut for first generating a wv = WindowViewer(x; kwargs...) and then applying mapped_f = map(f, wv). If x is accompanied by a time vector t, you probably also want to call this function with t instead of x and with one of mean, midpoint, midvalue as f to obtain a time vector for the mapped_f output.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.windowmap!","page":"API","title":"TransitionsInTimeseries.windowmap!","text":"windowmap!(f::Function, out, x::AbstractVector; kwargs...)\n\nSame as windowmap, but writes the output in-place in out.\n\n\n\n\n\n","category":"function"},{"location":"api/#Load-data","page":"API","title":"Load data","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"load_linear_vs_doublewell()","category":"page"},{"location":"api/#TransitionsInTimeseries.load_linear_vs_doublewell-Tuple{}","page":"API","title":"TransitionsInTimeseries.load_linear_vs_doublewell","text":"load_linear_vs_doublewell()\n\nLoad prototypical data from a linear and a double-well model to test some indicators.\n\n\n\n\n\n","category":"method"},{"location":"api/#Visualization","page":"API","title":"Visualization","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"plot_indicator_changes\nplot_significance!\nplot_changes_significance","category":"page"},{"location":"api/#TransitionsInTimeseries.plot_indicator_changes","page":"API","title":"TransitionsInTimeseries.plot_indicator_changes","text":"plot_indicator_changes(res) → (fig, axs)\n\nReturn fig::Figure and axs::Matrix{Axis}, on which res::ChangesResults has been visualised.\n\nKeyword arguments:\n\ncolors = default_colors sets the colors of the line plots that are to\n\nbe cycled through. The default correspond to the color scheme of Julia Dynamics.\n\n`indicatornames = defaultindicatorlabel(res), chametricnames =\n\ndefaultchametriclabel(res)sets the labels for the indicators and the change metrics, with the default inferring them from the names ofres.config.indicatorsandres.config.change_metrics`.\n\naccent_linewidth = 3 sets the line width for the original signals (the\n\nsurrogates have linewidth = 1)\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.plot_significance!","page":"API","title":"TransitionsInTimeseries.plot_significance!","text":"plot_significance!(axs, res, signif)\n\nUpdate the axs::Matrix{Axis} of a figure obtained with plot_indicator_changes(res) with the signif::SurrogatesSignificance.\n\nKeyword arguments:\n\nflags = nothing provides the significance flags, for instance obtained by\n\nthresholding the pvalues.\n\nnsurro = 20 sets the number of surrogates to plot.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.plot_changes_significance","page":"API","title":"TransitionsInTimeseries.plot_changes_significance","text":"plot_changes_significance(res, signif) → (fig, axs)\n\nReturn fig::Figure and axs::Matrix{Axis}, on which res::ChangesResults and signif::SurrogatesSignificance have been visualised. The source code is as simple as:\n\nfig, axs = plot_indicator_changes(res; kwargs...)\nplot_significance!(axs, res, signif; kwargs...)\n\nFor more information, refer to plot_indicator_changes and plot_significance!.\n\n\n\n\n\n","category":"function"},{"location":"api/#Utils","page":"API","title":"Utils","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"default_window_width","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"EditURL = \"do-events.jl\"","category":"page"},{"location":"examples/do-events/#Dansgaard-Oescher-events-and-Critical-Slowing-Down","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"The delta^18O timeseries of the North Greenland Ice Core Project (NGRIP) are, to this date, the best proxy record for the Dansgaard-Oeschger events (DO-events). DO-events are sudden warming episodes of the North Atlantic, reaching 10 degrees of regional warming within 100 years. They happened quasi-periodically over the last glacial cycle due to transitions between strong and weak states of the Atlantic Meridional Overturning Circulation and might be therefore be the most prominent examples of abrupt transitions in the field of climate science. We here propose to hindcast these events by applying the theory of Critical Slowing Down (CSD) on the NGRIP data, which can be found here in its raw format. This analysis has already been done in (Boers, 2018) and we here try to reproduce Figure 2.d-f.","category":"page"},{"location":"examples/do-events/#Preprocessing-NGRIP","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Preprocessing NGRIP","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"Data pre-processing is not part of TransitionsInTimeseries.jl, but a step the user has to do before using the package. To present an example with a complete scientific workflow, we will showcase typical data pre-processing here, that consist of the following steps:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"Load the data, reverse and offset it to have time vector = time before 2000 AD.\nFilter non-unique points in time and sort the data.\nRegrid the data from uneven to even sampling.","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"The time and delta^18O vectors resulting from the i-th preprocessing step are respectively called t_i and x_i. The final step consists in obtaining a residual r, i.e. the fluctuations of the system around the attractor, which, within the CSD theory, is assumed to be tracked. Over this example, it will appear that the convenience of TransitionsInTimeseries.jl leads the bulk of the code to be written for plotting and preprocessing.","category":"page"},{"location":"examples/do-events/#Step-1:","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Step 1:","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"using DelimitedFiles, Downloads\n\nfunction load_ngrip()\n tmp = Base.download(\"https://raw.githubusercontent.com/JuliaDynamics/JuliaDynamics/\"*\n \"master/timeseries/NGRIP.csv\")\n data, labels = readdlm(tmp, header = true)\n return reverse(data[:, 1]) .- 2000, reverse(data[:, 2]) # (time, delta-18-0) vectors\nend\n\nt1, x1 = load_ngrip()","category":"page"},{"location":"examples/do-events/#Step-2","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Step 2","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"uniqueidx(v) = unique(i -> v[i], eachindex(v))\nfunction keep_unique(t, x)\n unique_idx = uniqueidx(t)\n return t[unique_idx], x[unique_idx]\nend\n\nfunction sort_timeseries!(t, x)\n p = sortperm(t)\n permute!(t, p)\n permute!(x, p)\n return nothing\nend\n\nt2, x2 = keep_unique(t1, x1)\nsort_timeseries!(t2, x2)","category":"page"},{"location":"examples/do-events/#Step-3","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Step 3","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"using BSplineKit\n\nfunction regrid2evensampling(t, x, dt)\n itp = BSplineKit.interpolate(t, x, BSplineOrder(4))\n tspan = (ceil(minimum(t)), floor(maximum(t)))\n t_even = collect(tspan[1]:dt:tspan[2])\n x_even = itp.(t_even)\n return t_even, x_even\nend\n\ndt = 5.0 # dt = 5 yr as in (Boers 2018)\nt3, x3 = regrid2evensampling(t2, x2, dt)","category":"page"},{"location":"examples/do-events/#Step-4","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Step 4","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"For the final step we drop the indices:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"using DSP\n\nfunction chebyshev_filter(t, x, fcutoff)\n ii = 10 # Chebyshev filtering requires to prune first points of timeseries.\n responsetype = Highpass(fcutoff, fs = 1/dt)\n designmethod = Chebyshev1(8, 0.05)\n r = filt(digitalfilter(responsetype, designmethod), x)\n xtrend = x - r\n return t[ii:end], x[ii:end], xtrend[ii:end], r[ii:end]\nend\n\nfcutoff = 0.95 * 0.01 # cutoff ≃ 0.01 yr^-1 as in (Boers 2018)\nt, x, xtrend, r = chebyshev_filter(t3, x3, fcutoff)","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"Let's now visualize our data in what will become our main figure. For the segmentation of the DO-events, we rely on the tabulated data from (Rasmussen et al., 2014) (which will soon be available as downloadable):","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"using CairoMakie, Loess\n\nfunction loess_filter(t, x; span = 0.005)\n loessmodel = loess(t, x, span = span)\n xtrend = Loess.predict(loessmodel, t)\n r = x - xtrend\n return t, x, xtrend, r\nend\n\nfunction kyr_xticks(tticks_yr)\n tticks_kyr = [\"$t\" for t in Int.(tticks_yr ./ 1e3)]\n return (tticks_yr, tticks_kyr)\nend\n\nfunction plot_do(traw, xraw, tfilt, xfilt, t, r, t_transitions, xlims, xticks)\n fig = Figure(size = (1600, 1200), fontsize = 24)\n\n # Original timeseries with transition marked by vertical lines\n ax1 = Axis(fig[1, 1], xlabel = L\"Time (kyr) $\\,$\", ylabel = L\"$\\delta^{18}$O (permil)\",\n xaxisposition = :top, xticks = xticks)\n lines!(ax1, traw, xraw, color = (:gray70, 0.5))\n lines!(ax1, tfilt, xfilt, color = :gray10, linewidth = 3)\n vlines!(ax1, t_transitions, color = Cycled(1), linewidth = 3)\n\n # Residual timeseries\n ax2 = Axis(fig[2, 1], ylabel = L\"Residual $\\,$\", xticks = xticks,\n xticksvisible = false, xticklabelsvisible = false)\n lines!(ax2, t, r, color = :gray50, linewidth = 1)\n\n # Axes for variance and AC1 timeseries\n ax3 = Axis(fig[3, 1], ylabel = L\"Variance $\\,$\", xticks = xticks,\n xticksvisible = false, xticklabelsvisible = false)\n ax4 = Axis(fig[4, 1], xlabel = L\"Time (kyr) $\\,$\", ylabel = L\"Lag-1 autocor. $\\,$\",\n xticks = xticks)\n\n axs = [ax1, ax2, ax3, ax4]\n [xlims!(ax, xlims) for ax in axs]\n ylims!(axs[1], (-48, -34))\n rowgap!(fig.layout, 10)\n return fig, axs\nend\n\nxlims = (-60e3, -10e3)\nxticks = kyr_xticks(-60e3:5e3:5e3)\nt_rasmussen = -[-60000, 59440, 58280, 55800, 54220, 49280, 46860, 43340, 41460, 40160,\n 38220, 35480, 33740, 32500, 28900, 27780, 23340, 14692, 11703]\ntloess, _, xloess, rloess = loess_filter(t3, x3) # loess-filtered signal for visualization\nfig, axs = plot_do(t3, x3, tloess, xloess, t, r, t_rasmussen, xlims, xticks)\nfig","category":"page"},{"location":"examples/do-events/#Hindcast-on-NGRIP-data","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Hindcast on NGRIP data","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"As one can see... there is not much to see so far. Residuals are impossible to simply eye-ball and we therefore use TransitionsInTimeseries.jl to study the evolution, measured by the ridge-regression slope of the residual's variance and lag-1 autocorrelation (AC1) over time. In many examples of the literature, including (Boers, 2018), the CSD analysis is performed over segments (sometimes only one) of the timeseries, such that a significance value is obtained for each segment. By using SegmentedWindowConfig, dealing with segments can be easily done in TransitionsInTimeseries.jl and is demonstrated here:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"using TransitionsInTimeseries, StatsBase\nusing Random: Xoshiro\n\nac1(x) = sum(autocor(x, [1])) # AC1 from StatsBase\nindicators = (var, ac1)\nchange_metrics = (RidgeRegressionSlope(), RidgeRegressionSlope())\ntseg_start = t_rasmussen[1:end-1] .+ 200\ntseg_end = t_rasmussen[2:end] .- 200\nconfig = SegmentedWindowConfig(indicators, change_metrics,\n tseg_start, tseg_end; whichtime = last, width_ind = Int(200÷dt),\n min_width_cha = 100) # require >=100 data points to estimate change metric\nresults = estimate_changes(config, r, t)\nsignif = SurrogatesSignificance(n = 1000, tail = [:right, :right], rng = Xoshiro(1995))\nflags = significant_transitions(results, signif)","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"That's it! We can now visualise our results with a generic function that we will re-use later:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"function plot_segment_analysis!(axs, results, signif)\n (; t_indicator, x_indicator) = results\n for k in eachindex(t_indicator) # loop over the segments\n for i in axes(signif.pvalues, 2) # loop over the indicators\n if !isnan(signif.pvalues[k, i]) # plot if segment long enough\n # Plot indicator timeseries and its linear regression\n ti, xi = t_indicator[k], x_indicator[k][:, i]\n lines!(axs[i+2], ti, xi, color = Cycled(1))\n m, p = ridgematrix(ti, 0.0) * xi\n if signif.pvalues[k, i] < 0.05\n lines!(axs[i+2], ti, m .* ti .+ p, color = :gray5, linewidth = 3)\n else\n lines!(axs[i+2], ti, m .* ti .+ p, color = :gray60, linewidth = 3)\n end\n end\n end\n end\nend\nplot_segment_analysis!(axs, results, signif)\nfig","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"In (Boers, 2018), 13/16 and 7/16 true positives are respectively found for the variance and AC1, with 16 referring to the total number of transitions. The timeseries actually includes 18 transition but, in (Boers, 2018), some segments are considered too small to be analysed. In contrast, we here respectively find 9/16 true positives for the variance and 3/16 for AC1. We can track down the discrepancies to be in the surrogate testing, since the indicator timeseries computed here are almost exactly similar to those of (Boers, 2018). This mismatch points out that packages like TransitionsInTimeseries.jl are desirable for research to be reproducible, especially since CSD is gaining attention - not only within the scientific community but also in popular media.","category":"page"},{"location":"examples/do-events/#CSD:-only-a-necessary-condition,-only-in-some-cases","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"CSD: only a necessary condition, only in some cases","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"For codimension-1 systems, approaching a fold, Hopf or transcritical bifurcation implies a widening of the potential U, which defines the deterministic term f = -U of the SDE's right-hand-side. In the presence of noise, this leads to CSD, which is therefore a necessary condition for crossing one of these bifurcations - although it is not always assessable by analysing the timeseries due to practical limitations (e.g. sparse data subject to large measurement noise). It is nonetheless not given that DO-events, as many other real-life applications, can be seen as a codimension-1 fold, Hopf or transcritical bifurcations. Besides this, we emphasise that CSD is not a sufficient condition for assessing a transition being ahead in near future, since a resilience loss can happen without actually crossing any bifurcation. This can be illustrated on the present example by performing the same analysis only until few hundred years before the transition:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"tseg_end = t_rasmussen[2:end] .- 700 # stop analysis 500 years earlier than before\nconfig = SegmentedWindowConfig(indicators, change_metrics,\n tseg_start, tseg_end, whichtime = last, width_ind = Int(200÷dt),\n min_width_cha = 100)\nresults = estimate_changes(config, r, t)\nsignif = SurrogatesSignificance(n = 1000, tail = [:right, :right], rng = Xoshiro(1995))\nflags = significant_transitions(results, signif)\nfig, axs = plot_do(t3, x3, tloess, xloess, t, r, t_rasmussen, xlims, xticks)\nplot_segment_analysis!(axs, results, signif)\nfig","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"For the variance and AC1, we here respectively find 6 and 3 positives, although the transitions are still far ahead. This shows that what CSD captures is a potential widening induced by a shift of the forcing parameter rather than the actual transition. We therefore believe, as already suggested in some studies, that \"resilience-loss indicators\" is a more accurate name than \"early-warning signals\" when using CSD.","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"We draw attention upon the fact that the delta^18O timeseries is noisy and sparsely re-sampled. Furthermore, interpolating over time introduces a potential bias in the statistics, even if performed on a coarse grid. The NGRIP data therefore represents an example that should be handled with care - as many others where CSD analysis has been applied on transitions in the field of geoscience. To contrast with this, we propose to perform the same analysis on synthethic DO data, obtained from an Earth Model of Intermediate Complexity (EMIC).","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"warning: Degrees of freedom\nThese sources of error come along the usual problem of arbitrarily choosing (1) a filtering method, (2) windowing parameters and (3) appropriate metrics (for instance when the forcing noise is suspected to be correlated). This leads to a large number of degrees of freedom (DoF). Although sensible guesses are possible here, checking that results are robust w.r.t. the DoF should be a standard practice.","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"info: Future improvement\nSupporting the computations for uneven timeseries is a planned improvement of TransitionsInTimeseries.jl. This will avoid the need of regridding data on coarse grids and will prevent from introducing any bias.","category":"page"},{"location":"examples/do-events/#Hindcasting-simulated-DO-events","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Hindcasting simulated DO-events","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"In CLIMBER-X, the EMIC described in (Willeit et al., 2022), DO-like events can be triggered by forcing the North Atlantic with a (white noise) freshwater input. Simulated DO-like events present the big advantage of being evenly sampled in time and free of measurement noise. We run this analysis over two exemplary simulation outputs:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"t_transitions = [[1, 1850, 2970, 3970, 5070, 5810, 7050, 8050],\n [1, 3500, 4370, 5790, 7200, 8140]]\nt_lb = [[300, 500, 300, 600, 300, 500, 500], [1800, 500, 1000, 900, 500]]\ntseg_start = [t_transitions[1][1:end-1] + t_lb[1], t_transitions[2][1:end-1] + t_lb[2]]\ntseg_end = [t_transitions[1][2:end] .- 50, t_transitions[2][2:end] .- 50]\n\nfigvec = Figure[]\n\nfor j in 1:2\n # Download the data and perform loess filtering on it\n tmp = Base.download(\"https://raw.githubusercontent.com/JuliaDynamics/JuliaDynamics/\" *\n \"master/timeseries/climberx-do$(j)-omaxa.csv\")\n data = readdlm(tmp)\n tcx, xcx = data[1, 1000:end], data[2, 1000:end]\n t, x, xtrend, r = loess_filter(tcx, xcx, span = 0.02)\n\n # Initialize figure\n xlims = (0, last(tcx))\n xticks = kyr_xticks(xlims[1]:1e3:xlims[2])\n fig, axs = plot_do(tcx, xcx, t, xtrend, t, r, t_transitions[j], extrema(t), xticks)\n ylims!(axs[1], (5, 40))\n axs[1].ylabel = L\"Max. Atlantic overturning (Sv) $\\,$\"\n\n # Run sliding analysis and update figure with results\n dt = mean(diff(tcx))\n config = SegmentedWindowConfig(\n indicators, change_metrics, tseg_start[j], tseg_end[j],\n whichtime = last, width_ind = Int(200÷dt), min_width_cha = 100)\n results = estimate_changes(config, r, t)\n signif = SurrogatesSignificance(n = 1_000, tail = [:right, :right], rng = Xoshiro(1995))\n flags = significant_transitions(results, signif)\n\n plot_segment_analysis!(axs, results, signif)\n vlines!(axs[1], t_transitions[j], color = Cycled(1), linewidth = 3)\n push!(figvec, fig)\nend\nfigvec[1]","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"It here appears that not all transitions are preceeded by a significant increase of variance and AC1, even in the case of clean and evenly sampled time series. Let's check another case:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"figvec[2]","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"Same here! Although CLIMBER-X does not represent real DO-events, the above-performed analysis might be hinting at the fact that not all DO transitions can be forecasted with CSD. Nonetheless, performing a CSD analysis can inform on the evolution of a system's resilience.","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"EditURL = \"ks_paleojump.jl\"","category":"page"},{"location":"examples/ks_paleojump/#Kolmogorov-Smirnov-test-for-detecting-transitions-in-paleoclimate-timeseries","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"The goal of this example is to show how simple it is to re-create an analysis similar to what was done in the paper \"Automatic detection of abrupt transitions in paleoclimate records\", (Bagniewski et al., 2021). The same analysis was then used to create a database of transitions in paleoclimate records in (Bagniewski et al., 2023) Using TransitionsInTimeseries.jl and HypothesisTests.jl, the analysis becomes a 10-lines-of-code script (for a given timeseries).","category":"page"},{"location":"examples/ks_paleojump/#Scientific-background","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Scientific background","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"The approach of (Bagniewski et al., 2021) is based on the two-sample Kolmogorov Smirnov test. It tests whether the samples from two datasets or timeseries are distributed according to the same cumulative density function or not. This can be estimated by comparing the value of the KS-statistic versus some threshold that depends on the required confidence.","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"The application of this test for identifying transitions in timeseries is simple:","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"A sliding window analysis is performed in the timesries\nIn each window, the KS statistic is estimated between the first half and the second half of the timeseries within this window.\nTransitions are defined by when the KS statistic exceeds a particular value based on some confidence. The transition occurs in the middle of the window.","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"We should point out that in (Bagniewski et al., 2021) the authors did a more detailed analysis: analyzed many different window widths, added a conditional clause to exclude transitions that do not exceed a predefined minimum \"jump\" in the data, and also added another conditional clause that filtered out transitions that are grouped in time (which is a natural consequence of using the Kolmogorov-Smirov test for detecting transitions).","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"Here we won't do that post processing, mainly because it is rather simple to include these additional conditional clauses to filter transitions after they are found.","category":"page"},{"location":"examples/ks_paleojump/#Steps-for-TransitionsInTimeseries.jl","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Steps for TransitionsInTimeseries.jl","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"Doing this kind of work with TransitionsInTimeseries.jl is so easy you won't even trip! This analysis follows the same sliding window approach showcased in our Tutorial, and it even excludes the \"indicator\" aspect: the change metric is estimated directly from the input data!","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"As such, we really only need to define/do these things before we have finished the analysis:","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"Load the input data (we will use the same example as the NGRIP data of Dansgaard-Oescher events and Critical Slowing Down example) and set the appropriate time window.\nDefine the function that estimates the change metric (i.e., the KS-statistic)\nPerform the sliding window analysis as in the Tutorial with estimate_changes\nEstimate the \"confident\" transitions in the data by comparing the estimated KS-statistic with a predefined threshold.","category":"page"},{"location":"examples/ks_paleojump/#Load-timeseries-and-window-length","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Load timeseries and window length","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"Following the Dansgaard-Oescher events example, we load the data after all the processing steps done in that example:","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"using DelimitedFiles, CairoMakie\n\ntmp = Base.download(\"https://raw.githubusercontent.com/JuliaDynamics/JuliaDynamics/\"*\n \"master/timeseries/NGRIP_processed.csv\")\ndata = readdlm(tmp)\nt, xtrend, xresid, xloess = collect.(eachcol(data))\n\nfig, ax = lines(t, xtrend; axis = (ylabel = \"NGRIP (processed)\", xlabel = \"time\"))\nlines!(ax, t, xloess; linewidth = 2)\nfig","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"For the window, since we are using a sliding window here, we will be using a window of length 500 (which is approximately 1/2 to 1/4 the span between typical transitions found by (Rasmussen et al., 2014)).","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"window = 500","category":"page"},{"location":"examples/ks_paleojump/#Defining-the-change-metric-function","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Defining the change metric function","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"HypothesisTest.jl implements the Kolmogorov-Smirnov test, however here we are interested in the value of the test iself (the so-called KS-statistic), rather than a p-value. To this end, we define the following function to compute the statistic, which also normalizes it as in (Bagniewski et al., 2021).","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"using HypothesisTests\n\nfunction normalized_KS_statistic(timeseries)\n N = length(timeseries)\n i = N÷2\n x = view(timeseries, 1:i)\n y = view(timeseries, (i+1):N)\n kstest = ApproximateTwoSampleKSTest(x, y)\n nx = ny = i # length of each timeseries half of total\n n = nx*ny/(nx + ny) # written fully for concreteness\n D_KS = kstest.δ # can be compared directly with sqrt(-log(α/2)/2)\n # Rescale according to eq. (5) of the paper\n rescaled = 1 - ((1 - D_KS)/(1 - sqrt(1/n)))\n return rescaled\nend\n\nN = 1000 # the statistic is independent of `N` for large enough `N`!\nx = randn(N)\ny = 1.8randn(N) .+ 1.0\nz = randn(N)\nw = 0.6randn(N) .- 2.0\n\nfig, ax = density(x; color = (\"black\", 0.5), strokewidth = 4.0, label = \"reference distribution\")\nax.title = \"showcase of normalized KS-statistic\"\nfor q in (y, z, w)\n D_KS = normalized_KS_statistic(vcat(x, q))\n density!(ax, q; label = \"D_KS = $(D_KS)\")\nend\naxislegend(ax)\nfig","category":"page"},{"location":"examples/ks_paleojump/#Perform-the-sliding-window-analysis","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Perform the sliding window analysis","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"This is just a straightforward call to estimate_changes. In fact, it is even simpler than the tutorial. Here we skip completely the \"indicator\" estimation step, and we evaluate the change metric directly on input data. We do this by simply passing nothing as the indicators.","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"using TransitionsInTimeseries\n\nconfig = SlidingWindowConfig(nothing, normalized_KS_statistic; width_cha = 500)\n\nresults = estimate_changes(config, xtrend, t)","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"Which we can visualize","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"function visualize_results(results)\n fig, ax1 = lines(t, xtrend; axis = (ylabel = \"NGRIP (processed)\",))\n ax2, = lines(fig[2, 1], results.t_change, vec(results.x_change), axis = (ylabel = \"D_KS (normalized)\", xlabel = \"time\"))\n linkxaxes!(ax1, ax2)\n hidexdecorations!(ax1; grid = false)\n xloess_normed = (xloess .- minimum(xloess))./(maximum(xloess) - minimum(xloess))\n lines!(ax2, t, xloess_normed; color = (\"gray\", 0.5))\n fig\nend\n\nvisualize_results(results)","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"By overplotting the (smoothened) NGRIP timeseries and the normalized KS-statistic, it already becomes pretty clear that the statistic peaks when transitions occur.","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"The same thing happens if we alter the window duration","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"config = SlidingWindowConfig(nothing, normalized_KS_statistic; width_cha = 200)\nresults = estimate_changes(config, xtrend, t)\nvisualize_results(results)","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"So one can easily obtain extra confidence by varying window size as in (Bagniewski et al., 2021).","category":"page"},{"location":"examples/ks_paleojump/#Identifying-\"confident\"-transitions","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Identifying \"confident\" transitions","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"As this identification here is done via a simple threshold, identifying the transitions is a nearly trivial call to significant_transitions with ThresholdSignificance","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"signif = ThresholdSignificance(0.5) # or any other threshold\nflags = significant_transitions(results, signif)\n\nfig = visualize_results(results)\naxDKS = content(fig[2,1])\nvlines!(axDKS, results.t_change[vec(flags)], color = (\"red\", 0.25))\nfig","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"We could proceed with a lot of preprocessing as in (Bagniewski et al., 2021) but we skip this here for the sake of simplicity.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"EditURL = \"logistic.jl\"","category":"page"},{"location":"examples/logistic/#Permutation-entropy-for-dynamic-regime-changes","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Permutation entropy is used frequently to detect a transition between one dynamic regime to another. It is useful when the mean and std. of the timeseries values are very similar between the two regimes, which would mean that common distribution-based indicators, or common critical-slowing-down based indicators, would fail.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"This example will also explore different ways to test for significance that are arguably better suitable in such an application than the Tutorial's default of significance via random Fourier surrogates.","category":"page"},{"location":"examples/logistic/#Logistic-map-timeseries","page":"Permutation entropy for dynamic regime changes","title":"Logistic map timeseries","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"A simple example of this is transitions from periodic to weakly chaotic to chaotic motion in the logistic map. First, let's generate a timeseries of the logistic map","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"using DynamicalSystemsBase\nusing CairoMakie\n\n# time-dependent logistic map, so that the `r` parameter increases with time\nr1 = 3.83\nr2 = 3.86\nN = 2000\nrs = range(r1, r2; length = N)\n\nfunction logistic_drifting_rule(u, rs, n)\n r = rs[n+1] # time is `n`, starting from 0\n return SVector(r*u[1]*(1 - u[1]))\nend\n\nds = DeterministicIteratedMap(logistic_drifting_rule, [0.5], rs)\nx = trajectory(ds, N-1)[1][:, 1]","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Plot it, using as time the parameter value (they coincide)","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"fig, ax = lines(rs, x; linewidth = 0.5)\nax.xlabel = \"r (time)\"\nax.ylabel = \"x\"\nfig","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"In this example there is a rather obvious transition to strongly chaotic motion at r ≈ 3.855. However, there is also a subtle transition to weak chaos at r ≈ 3.847. This transition is barely visible in the timeseries, and in fact many of the timeseries statistical properties remain identical.","category":"page"},{"location":"examples/logistic/#Using-a-simpler-change-metric","page":"Permutation entropy for dynamic regime changes","title":"Using a simpler change metric","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Now, let's compute various indicators and their changes, focusing on the permutation entropy as an indicator. We use order 4 here, because we know that to detect changes in a period m we would need an order ≥ m+1 permutation entropy.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"using TransitionsInTimeseries, ComplexityMeasures\n\nfunction permutation_entropy(m)\n est = SymbolicPermutation(; m) # order 3\n indicator = x -> entropy_normalized(est, x)\n return indicator\nend\n\nindicators = (var, ar1_whitenoise, permutation_entropy(4))\nindistrings = (\"var\", \"ar1\", \"pe\")","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"In this example there is no critical slowing down; instead, there is a sharp transition between periodic and chaotic motion. Hence, we shouldn't be using any trend-based change metrics. Instead, we will use the most basic change metric, difference_of_means. With this metric it also makes most sense to use as stride half the window width","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"metric = (difference_of_means, difference_of_means, difference_of_means)\n\nwidth_ind = N÷100\nwidth_cha = 20\nstride_cha = 10\n\nconfig = SlidingWindowConfig(indicators, metric;\n width_ind, width_cha, stride_cha,\n)\n\nresults = estimate_changes(config, x, rs)","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Let's now plot the change metrics of the indicators","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"function plot_change_metrics()\n fig, ax = lines(rs, x; axis = (ylabel = \"input\",), figure = (size = (600, 600),))\n hidexdecorations!(ax; grid = false)\n # plot all change metrics\n for (i, c) in enumerate(eachcol(results.x_change))\n ax, = scatterlines(fig[i+1, 1], results.t_change, c;\n axis = (ylabel = indistrings[i],), label = \"input\"\n )\n if i < 3\n hidexdecorations!(ax; grid = false)\n else\n ax.xlabel = \"r (time)\"\n end\n end\n return fig\nend\n\nfig = plot_change_metrics()","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"We already see the interesting results we expect: the permutation entropy shows a striking change as we go from periodic to weakly chaotic motion at r ≈ 3.847. (Remember: the plotted quantity is how much the indicator changes within a time window. High values mean large changes.)","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Due to its construction, permutation entropy will have a spike for periodic data at the start of the timeseries, so we can safely ignore the spike at r ≈ 3.83.","category":"page"},{"location":"examples/logistic/#Significance-via-random-Fourier-surrogates","page":"Permutation entropy for dynamic regime changes","title":"Significance via random Fourier surrogates","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"One way to test for significance would be via the standard way as in the Tutorial, utilizing surrogate timeseries and SurrogatesSignificance.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Let's do it here for an example, but, we have to be careful. It is crucial that for permutation entropy we use :right as the tail, because it is expected that the surrogates will have higher differences in the permutation entropy timeseries (because, if there is no dynamical change, the permutation entropy will stay the same, while in the surrogates there are always random fluctuations!","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"surromethod = RandomFourier()\n\n# Define a function because we will re-use later\nusing Random: Xoshiro\nfunction overplot_surrogate_significance!(fig, surromethod, color = \"black\")\n\n signif = SurrogatesSignificance(;\n n = 1000, tail = [:both, :both, :right], surromethod, rng = Xoshiro(42),\n )\n flags = significant_transitions(results, signif)\n\n # and also plot the flags with same color\n for (i, indicator) in enumerate(indicators)\n # To make things visually clear, we will also plot some example surrogate\n # timeseries for each indicator and change metric pair\n for _ in 1:10\n s = TimeseriesSurrogates.surrogate(x, surromethod)\n p = windowmap(indicator, s; width = width_ind)\n q = windowmap(metric[i], p; width = width_cha, stride = stride_cha)\n lines!(fig[i+1, 1], results.t_change, q; color = (color, 0.2), linewidth = 1)\n end\n # Plot the flags as vertical dashed lines\n vlines!(fig[i+1, 1], results.t_change[flags[:, i]];\n color = color, linestyle = :dash, linewidth = 3\n )\n end\n # add a title to the figure with how we estimate significance\n content(fig[1, 1]).title = \"surrogates: \"*string(nameof(typeof(surromethod)))\nend\n\nsurromethod = RandomFourier()\noverplot_surrogate_significance!(fig, surromethod)\n\nfig","category":"page"},{"location":"examples/logistic/#Different-surrogates","page":"Permutation entropy for dynamic regime changes","title":"Different surrogates","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Random Fourier surrogates perserve the power spectrum of the timeseries, but the power spectrum is a property integrated over the whole timeseries. It doesn't contain any information highlighting the local dynamics or information that preserves the local changes of dynamical behavior.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"A surrogate type that does a better job in preserving local sharp changes in the timeseries (and hence provides stricter surrogate-based significance) is for example RelativePartialRandomization.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"A much better alternative is to use block-shuffled surrogates, which preserve the short term local temporal correlation in the timeseries and hence also preserve local short term sharp changes in the dynamic behavior.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"surromethod = RelativePartialRandomization(0.25)\nfig = plot_change_metrics()\noverplot_surrogate_significance!(fig, surromethod, \"gray\")\nfig","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Our results have improved. In the permutation entropy, we see only two transitions detected as significant, which is correct: only two real dynamical transitions exist in the data. In the other two indicators we also see fewer transitions, but as we have already discussed, no results with the other indicators should be taken into meaningful consideration, as these indicators are simply inappropriate for what we are looking for here.","category":"page"},{"location":"examples/logistic/#Simpler-Significance","page":"Permutation entropy for dynamic regime changes","title":"Simpler Significance","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Arguably, exactly because we are using the difference_of_means as a change metric, we may want to be less strict and more simple with our tests for significance. Instead of using SurrogatesSignificance we may use the simpler and much faster SigmaSignificance, which simply claims significant time points whenever a change metric exceeds some pre-defined factor of its timeseries standard deviation.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"fig = plot_change_metrics()\nflags = significant_transitions(results, SigmaSignificance(factor = 5.0))\n\n# Plot the flags\nfor (i, indicator) in enumerate(indicators)\n vlines!(fig[i+1, 1], results.t_change[flags[:, i]];\n color = Cycled(3), linestyle = :dash, linewidth = 3\n )\nend\ncontent(fig[1, 1]).title = \"significance from std\"\nfig","category":"page"},{"location":"#TransitionsInTimeseries.jl","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"","category":"section"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"(Image: TransitionsInTimeseries.jl)","category":"page"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"TransitionsInTimeseries","category":"page"},{"location":"#TransitionsInTimeseries","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries","text":"TransitionsInTimeseries.jl\n\n(Image: ) (Image: ) (Image: CI) (Image: codecov) (Image: Package Downloads)\n\nTransitionsInTimeseries.jl is a free and open-source software to easily analyse transitions within timeseries in a reproducible, performant, extensible and reliable way. In contrast to other existing software with similar target application, TransitionsInTimeseries.jl defines a generic interface for how to find transitions and how to test for significance. Within this interface, it is easy to expand the software in three orthogonal ways:\n\nProvide the analysis pipelines with new indicators, which can be either self-written or imported from other packages. In particular, the latter offers thousands of metrics that can indicate transitions right out of the box.\nAdd new analysis pipelines for finding transitions.\nAdd new ways for significance testing.\n\nTransitionsInTimeseries is a registered Julia package and can be installed by running:\n\n] add TransitionsInTimeseries\n\nAll further information is provided in the documentation, which you can either find online or build locally by running the docs/make.jl file.\n\nAlternative names for this package could have been: Early Warning Signals / Resilience Indicators / Regime-Shift Identifiers / Change-Point Detectors, or however else you want to call them!\n\n\n\n\n\n","category":"module"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"info: Star us on GitHub!\nIf you have found this package useful, please consider starring it on GitHub. This gives us an accurate lower bound of the (satisfied) user count.","category":"page"},{"location":"#content","page":"TransitionsInTimeseries.jl","title":"Content","text":"","category":"section"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"Multi-stable systems can display abrupt transitions between two stability regimes. To predict such transitions in real-world systems solely based on data, mathematical tools have been developed in the last decades. Numerous terminologies have been used for them, such as early warning signals, resilience indicators, regime-shift identifiers, change-point detection and transition indicators. TransitionsInTimeseries.jl sticks to the latter terminology and provides an interface that:","category":"page"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"Allows a fast computation of common transition indicators with a couple of lines, as demonstrated in the example section.\nMakes the surrogate analysis to test for significance under the hub.\nCan be easily extended by any user without touching the source code.\nReduces the programming overhead for any researcher willing to benchmark new methods.\nEases the reproducibility thanks to a clear syntax, a simple installation and RNG-seeded surrogate generation.\nIncreases trustworthiness thanks to a large test suite.","category":"page"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"info: Similar projects\nAn R toolbox and a Python library already exist. However, we believe that they are difficult to extend for the user. Furthermore, they do not offer a native performant code, as here allowed by the use of Julia.","category":"page"},{"location":"#approaches","page":"TransitionsInTimeseries.jl","title":"Approaches","text":"","category":"section"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"Over the last decades, research on transition indicators has largely focused on Critical Slowing Down (CSD). CSD is observed when a system approaches a Hopf, a transcritical or a fold bifurcation and consists in a resilience loss of the system. For instance this can be diagnosed by an increase of the variance and the AR1-regression coefficient, as demonstrated in the example section. However, we emphasize that this is one out of many possible approaches for obtaining transition indicators. Recent work has explored new approaches relying on nonlinear dynamics or machine learning. TransitionsInTimeseries.jl is designed to allow these cutting-edge methods and foster the development of new ones.","category":"page"},{"location":"devdocs/#Developer's-documentation","page":"Developer's documentation","title":"Developer's documentation","text":"","category":"section"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"This documentation addresses users that would like to contribute to the software by either solving bugs, improving documentation, or adding new methods. All contributions come in the form of Pull Requests, for which we strongly advise to follow good practices in scientific code, which means that they are properly formatted, documented, tested, etc.","category":"page"},{"location":"devdocs/#New-indicators-or-change-metrics","page":"Developer's documentation","title":"New indicators or change metrics","text":"","category":"section"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"As explained already in e.g., SlidingWindowConfig, new indicators or change metrics are standard Julia functions, so you only need to define such a function (and document it, test it, etc.).","category":"page"},{"location":"devdocs/#New-pipeline-for-estimating-changes","page":"Developer's documentation","title":"New pipeline for estimating changes","text":"","category":"section"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"This means to contribute a fundamentally new \"pipeline\" for estimating/detecting transitions in timeseries. This new pipeline defines what a \"transition\" means. To add a new pipeline follow these steps:","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"Define a new subtype of ChangesConfig\nDefine a new subtype of ChangesResults\nAdd a method for estimate_changes which accepts the new ChangesConfig subtype you defined and returns the ChangesResults subtype you defined.","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"And that's it!","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"Optionally you can further extend:","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"TransitionsInTimeseries.plot_indicator_changes with the new ChangesResults type you defined. Note the plotting functions are in the ext folder of the repository.\nsignificant_transitions(res::ChangesResults, signif::Significance) with your new ChangesResults type and as many Significance subtypes you have the capacity to extend for.","category":"page"},{"location":"devdocs/#New-pipeline-for-estimating-significance","page":"Developer's documentation","title":"New pipeline for estimating significance","text":"","category":"section"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"Statistically significant changes are \"transitions\". However, what \"significant\" means is not universal. There are different ways to test for significance and TransitionsInTimeseries.jl allows various methods.","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"To add a new pipeline for estimating significance follow these steps:","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"Define a new subtype of Significance\nExtend significant_transitions(res::ChangesResults, signif::Significance) with your new type and as many ChangesResults subtypes as you have capacity for.","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"And that's it! Unfortunately so far we have found no way to make the significant_transitions code agnostic of the changes result, so you would need to add a method manually for every ChangesResults.","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"Optionally you can further extend TransitionsInTimeseries.plot_significance! (which you can find in the ext folder) for visualizing the significance results.","category":"page"},{"location":"refs/#References","page":"References","title":"References","text":"","category":"section"},{"location":"refs/","page":"References","title":"References","text":"Bagniewski, W.; Ghil, M. and Rousseau, D. D. (2021). Automatic detection of abrupt transitions in paleoclimate records. Chaos 31.\n\n\n\nBagniewski, W.; Rousseau, D. D. and Ghil, M. (2023). The PaleoJump database for abrupt transitions in past climates. Scientific Reports 13, 1–18, arXiv:2206.06832.\n\n\n\nBoers, N. (2018). Early-warning signals for Dansgaard-Oeschger events in a high-resolution ice core record. Nature Communications 9, 2556. Accessed on Oct 28, 2022.\n\n\n\nRasmussen, S. O.; Bigler, M.; Blockley, S. P.; Blunier, T.; Buchardt, S. L.; Clausen, H. B.; Cvijanovic, I.; Dahl-Jensen, D.; Johnsen, S. J.; Fischer, H.; Gkinis, V.; Guillevic, M.; Hoek, W. Z.; Lowe, J. J.; Pedro, J. B.; Popp, T.; Seierstad, I. K.; Steffensen, J. P.; Svensson, A. M.; Vallelonga, P.; Vinther, B. M.; Walker, M. J.; Wheatley, J. J. and Winstrup, M. (2014). A stratigraphic framework for abrupt climatic changes during the Last Glacial period based on three synchronized Greenland ice-core records: refining and extending the INTIMATE event stratigraphy. Quaternary Science Reviews 106, 14–28. Accessed on Oct 4, 2023.\n\n\n\nWilleit, M.; Ganopolski, A.; Robinson, A. and Edwards, N. R. (2022). The Earth system model CLIMBER-X v1.0 – Part 1: Climate model description and validation​​​​​​​​​​​​​​. Geoscientific Model Development 15, 5905–5948. Accessed on Aug 11, 2022.\n\n\n\n","category":"page"}] +[{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"EditURL = \"tutorial.jl\"","category":"page"},{"location":"tutorial/#Tutorial","page":"Tutorial","title":"Tutorial","text":"","category":"section"},{"location":"tutorial/#workflow","page":"Tutorial","title":"Workflow","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Computing transition indicators consists of the following steps:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Doing any preprocessing of raw data first, such as detrending. This not part of TransitionsInTimeseries.jl and yields the input timeseries.\nEstimating the timeseries of an indicator by sliding a window over the input timeseries.\nComputing the changes of the indicator by sliding a window over its timeseries. Alternatively, the change metric can be estimated over the whole segment, as examplified in the section Segmented windows.\nGenerating many surrogates that preserve important statistical properties of the original timeseries.\nPerforming step 2 and 3 for the surrogate timeseries.\nChecking whether the indicator change timeseries of the real timeseries shows a significant feature (trend, jump or anything else) when compared to the surrogate data.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"These steps are illustrated one by one in the tutorial below, and then summarized in the convenient API that TransitionsInTimeseries.jl exports.","category":"page"},{"location":"tutorial/#example_stepbystep","page":"Tutorial","title":"Tutorial – Educational","text":"","category":"section"},{"location":"tutorial/#Raw-input-data","page":"Tutorial","title":"Raw input data","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Let us load data from a bistable nonlinear model subject to noise and to a gradual change of the forcing that leads to a transition. Furthermore, we also load data from a linear model, which is by definition monostable and therefore incapable of transitioning. This is done to control the rate of false positives, a common problem that can emerge when looking for transition indicators. The models are governed by:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"dfracmathrmdx_lmathrmdt = - x_l - 1 + f(t) + n(t) \ndfracmathrmdx_nlmathrmdt = - x_nl^3 + x_nl + f(t) + n(t)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"with x_l the state of the linear model, x_nl the state of the bistable model, f the forcing and n the noise. For f=0 they both display an equilibrium point at x=-1. However, the bistable model also displays a further equilibrium point at x=1. Loading (and visualizing with Makie) such prototypical data to test some indicators can be done by simply running:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using TransitionsInTimeseries, CairoMakie\n\nt, x_linear, x_nlinear = load_linear_vs_doublewell()\nfig, ax = lines(t, x_linear)\nlines!(ax, t, x_nlinear)\nax.title = \"raw data\"\nfig","category":"page"},{"location":"tutorial/#Preprocessing","page":"Tutorial","title":"Preprocessing","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"note: Not part of TransitionsInTimeseries.jl\nAny timeseries preprocessing, such as the de-trending step we do here, is not part of TransitionsInTimeseries.jl and is the responsibility of the researcher.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The nonlinear system clearly displays a transition between two stability regimes. To forecast such transition, we analyze the fluctuations of the timeseries around the attractor, assumed to be tracked. Therefore, a detrending step is needed - here simply obtained by building the difference of the timeseries with lag 1.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"x_l_fluct = diff(x_linear)\nx_nl_fluct = diff(x_nlinear)\ntfluct = t[2:end]\n\nfig, ax = lines(tfluct, x_l_fluct)\nlines!(ax, tfluct, x_nl_fluct .+ 0.05)\nax.title = \"input timeseries\"\nfig","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"At this point, x_l_fluct and x_nl_fluct are considered the input timeseries.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"info: Detrending in Julia\nDetrending can be performed in many ways. A wide range of Julia packages exists to perform smoothing such as Loess.jl or DSP.jl. There the detrending step consists of subtracting the smoothed signal from the original one.","category":"page"},{"location":"tutorial/#Indicator-timeseries","page":"Tutorial","title":"Indicator timeseries","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"We can then compute the values of some \"indicator\" (a Julia function that inputs a timeseries and outputs a number). An indicator should be a quantity that is likely to change if a transition occurs, or is about to occur in the timeseries. We compute indicators by applying a sliding window over the input timeseries, determined by the width and the stride with which it is applied. Here we demonstrate this computation with the AR1-regression coefficient (under white-noise assumption), implemented as ar1_whitenoise:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"indicator = ar1_whitenoise\nindicator_window = (width = 400, stride = 1)\n\n# By mapping `last::Function` over a windowviewer of the time vector,\n# we obtain the last time step of each window.\n# This therefore only uses information from `k-width+1` to `k` at time step `k`.\n# Alternatives: `first::Function`, `midpoint:::Function`.\nt_indicator = windowmap(last, tfluct; indicator_window...)\nindicator_l = windowmap(indicator, x_l_fluct; indicator_window...)\nindicator_nl = windowmap(indicator, x_nl_fluct; indicator_window...)\n\nfig, ax = lines(t_indicator, indicator_l)\nlines!(ax, t_indicator, indicator_nl)\nax.title = \"indicator timeseries\"\nfig","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The lines plotted above are the indicator timeseries.","category":"page"},{"location":"tutorial/#Change-metric-timeseries","page":"Tutorial","title":"Change metric timeseries","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"From here, we process the indicator timeseries to quantify changes in it. This step is in essence the same as before: we apply some function over a sliding window of the indicator timeseries. We call this new timeseries the change metric timeseries. In the example here, the change metric we will employ will be the slope (over a sliding window), calculated via means of a RidgeRegressionSlope:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"change_window = (width = 30, stride = 1)\nridgereg = RidgeRegressionSlope(lambda = 0.0)\nprecompridgereg = precompute(ridgereg, t[1:change_window.width])\n\nt_change = windowmap(last, t_indicator; change_window...)\nchange_l = windowmap(precompridgereg, indicator_l; change_window...)\nchange_nl = windowmap(precompridgereg, indicator_nl; change_window...)\n\nfig, ax = lines(t_change, change_l)\nlines!(ax, t_change, change_nl)\nax.title = \"change metric timeseries\"\nfig","category":"page"},{"location":"tutorial/#Timeseries-surrogates","page":"Tutorial","title":"Timeseries surrogates","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"As expected from Critical Slowing Down, an increase of the AR1-regression coefficient can be observed. Although eyeballing the timeseries might already be suggestive, we want a rigorous framework for testing for significance.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"In TransitionsIdentifiers.jl we perform significance testing using the method of timeseries surrogates and the TimeseriesSurrogates.jl Julia package. This has the added benefits of reproducibility, automation and flexibility in choosing the surrogate generation method. Note that TimeseriesSurrogates is re-exported by TransitionsInTimeseries, so that you don't have to using both of them.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"To illustrate the surrogate, we compare the change metric computed from the bistable timeseries what that computed from a surrogate of the same timeseries.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"# Generate Fourier random-phase surrogates\nusing Random: Xoshiro\ns = surrogate(x_nl_fluct, RandomFourier(), Xoshiro(123))\n\nfunction gridfig(nrows, ncols)\n fig = Figure()\n axs = [Axis(fig[i, j], xticklabelsvisible = i == nrows ? true : false)\n for j in 1:ncols, i in 1:nrows]\n rowgap!(fig.layout, 10)\n return fig, axs\nend\nfig, axs = gridfig(2, 1)\nlines!(axs[1], tfluct, x_nl_fluct, color = Cycled(2))\nlines!(axs[1], tfluct, s .- 0.05, color = Cycled(3))\naxs[1].title = \"real signal vs. surrogate(s)\"\n\n# compute and plot indicator and change metric\nindicator_s = windowmap(indicator, s; indicator_window...)\nchange_s = windowmap(precompridgereg, indicator_s; change_window...)\n\nlines!(axs[2], t_change, change_nl, label = \"nonlin\", color = Cycled(2))\nlines!(axs[2], t_change, change_s, label = \"surrogate\", color = Cycled(3))\naxislegend()\naxs[2].title = \"change metric\"\n\n[xlims!(ax, 0, 50) for ax in axs]\nfig","category":"page"},{"location":"tutorial/#Quantifying-significance","page":"Tutorial","title":"Quantifying significance","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"To quantify the significance of the values of the change metric timeseries we perform a standard surrogate test by computing the p-value w.r.t. the change metrics of thousands of surrogates of the input timeseries. A low p-value (typically p<0.05) is commonly considered as significant. To visualize significant trends, we plot the p-value vs. time:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"n_surrogates = 1_000\nfig, axs = gridfig(2, 2)\naxs[1].title = \"linear\"\naxs[2].title = \"nonlinear\"\n\nfor (j, ax, axsig, x) in zip(1:2, axs[1:2], axs[3:4], (x_l_fluct, x_nl_fluct))\n\n orig_change = j == 1 ? change_l : change_nl\n sgen = surrogenerator(x, RandomFourier(), Xoshiro(123))\n pval = zeros(length(change_s))\n\n # Collect all surrogate change metrics\n for i in 1:n_surrogates\n s = sgen()\n indicator_s = windowmap(indicator, s; indicator_window...)\n change_s = windowmap(precompridgereg, indicator_s; change_window...)\n pval += orig_change .< change_s\n end\n\n pval ./= n_surrogates\n lines!(ax, t_change, orig_change) # ; color = Cycled(j)\n lines!(axsig, t_change, pval) # ; color = Cycled(j+2)\nend\n\n[xlims!(ax, 0, 50) for ax in axs]\nfig","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"As expected, the data generated by the nonlinear model displays a significant increase of the AR1-regression coefficient before the transition, which is manifested by a low p-value. In contrast, the data generated by the linear model does not show anything similar.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Performing the step-by-step analysis of transition indicators is possible and might be preferred for users wanting high flexibility. However, this results in a substantial amount of code. We therefore provide convenience functions that wrap this analysis, as shown in the next section.","category":"page"},{"location":"tutorial/#example_fastforward","page":"Tutorial","title":"Tutorial – TransitionsInTimeseries.jl","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"TransitionsInTimeseries.jl wraps this typical workflow into a simple, extendable, and modular API that researchers can use with little effort. In addition, it allows performing the same analysis for several indicators / change metrics in one go.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The interface is simple, and directly parallelizes the Workflow. It is based on the creation of a ChangesConfig, which contains a list of indicators, and corresponding metrics, to use for doing the above analysis. It also specifies what kind of surrogates to generate.","category":"page"},{"location":"tutorial/#Sliding-windows","page":"Tutorial","title":"Sliding windows","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The following blocks illustrate how the above extensive example is re-created in TransitionsInTimeseries.jl","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"using TransitionsInTimeseries, CairoMakie\n\nt, x_linear, x_nlinear = load_linear_vs_doublewell()\n\n# input timeseries and time\ninput = x_nl_fluct = diff(x_nlinear)\nt = t[2:end]\n\nfig, ax = lines(t, input)\nax.title = \"input timeseries\"\nfig","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"To perform all of the above analysis we follow a 2-step process.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Step 1, we decide what indicators and change metrics to use in SlidingWindowConfig and apply those via a sliding window to the input timeseries using estimate_changes.","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"# These indicators are suitable for Critical Slowing Down\nindicators = (var, ar1_whitenoise)\n\n# use the ridge regression slope for both indicators\nchange_metrics = (RidgeRegressionSlope(), RidgeRegressionSlope())\n\n# choices go into a configuration struct\nconfig = SlidingWindowConfig(indicators, change_metrics;\n width_ind = 400, width_cha = 30, whichtime = last)\n\n# choices are processed\nresults = estimate_changes(config, input, t)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"We can conveniently plot the information contained in results by using plot_indicator_changes:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"fig = plot_indicator_changes(results)","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"Step 2 is to estimate significance using SurrogatesSignificance and the function significant_transitions. Finally, we can conveniently plot the results obtained by updating the figure obtained above with plot_significance!:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"signif = SurrogatesSignificance(n = 1000, tail = [:right, :right])\nflags = significant_transitions(results, signif)\nplot_significance!(fig, results, signif, flags = flags)\nfig","category":"page"},{"location":"tutorial/#segmented_windows","page":"Tutorial","title":"Segmented windows","text":"","category":"section"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"The analysis shown so far relies on sliding windows of the change metric. This is particularly convenient for transition detection tasks. Segmented windows for the change metric computation are however preferable when it comes to prediction tasks. By only slightly modifying the syntax used so far, one can perform the same computations on segmented windows, as well as visualise the results conveniently:","category":"page"},{"location":"tutorial/","page":"Tutorial","title":"Tutorial","text":"config = SegmentedWindowConfig(indicators, change_metrics,\n t[1:1], t[1200:1200]; whichtime = last, width_ind = 200,\n min_width_cha = 100)\nresults = estimate_changes(config, input, t)\nsignif = SurrogatesSignificance(n = 1000, tail = [:right, :right])\nflags = significant_transitions(results, signif)\nfig = plot_changes_significance(results, signif)","category":"page"},{"location":"api/#API","page":"API","title":"API","text":"","category":"section"},{"location":"api/#Main-analysis-functions","page":"API","title":"Main analysis functions","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"estimate_changes\nChangesConfig\nChangesResults","category":"page"},{"location":"api/#TransitionsInTimeseries.estimate_changes","page":"API","title":"TransitionsInTimeseries.estimate_changes","text":"estimate_changes(config::ChangesConfig, x [,t]) → result\n\nEstimate possible transitions for input timeseries x using the approach specified in the configuration type config, see ChangesConfig for possibilities. t is the time vector corresponding to x, which defaults to eachindex(x).\n\nReturn the output as subtype of ChangesResults. The particular form of the output depends on the config and is described in its docstring. Regardless of type, result can always be given to significant_transitions to deduce which possible transitions are statistically significant using a variety of significance tests.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.ChangesConfig","page":"API","title":"TransitionsInTimeseries.ChangesConfig","text":"ChangesConfig\n\nSupertype for how \"changes\" in a timeseries are estimated in estimate_changes. \"Changes\" deemed statistically significant in significant_transitions are \"transitions\" in the timeseries.\n\nExisting subtypes of ChangesConfig are:\n\nSlidingWindowConfig.\nSegmentedWindowConfig.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.ChangesResults","page":"API","title":"TransitionsInTimeseries.ChangesResults","text":"ChangesResults\n\nSupertype used to gather results of estimate_changes. The concrete subtype instances are described in the docstrings of configuration types.\n\n\n\n\n\n","category":"type"},{"location":"api/#Sliding-window","page":"API","title":"Sliding window","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"SlidingWindowConfig\nSlidingWindowResults","category":"page"},{"location":"api/#TransitionsInTimeseries.SlidingWindowConfig","page":"API","title":"TransitionsInTimeseries.SlidingWindowConfig","text":"SlidingWindowConfig <: ChangesConfig\nSlidingWindowConfig(indicators, change_metrics; kwargs...)\n\nA configuration that can be given to estimate_changes. It estimates transitions by a sliding window approach:\n\nEstimate the timeseries of an indicator by sliding a window over the input timeseries.\nEstimate changes of an indicator by sliding a window of the change metric over the indicator timeseries.\n\nBoth indicators and change metrics are generic Julia functions that input an x::AbstractVector and output an s::Real. Any function may be given and see making custom indicators/change metrics in the documentation for more information on possible optimizations.\n\nindicators can be a single function or a tuple of indicators. Similarly, change_metrics can be a tuple or a single function. If tuples, the length of indicators and change_metrics must match. This way the analysis can be efficiently repeated for many indicators and/or change metrics.\n\nThe results output corresponding to SlidingWindowConfig is SlidingWindowResults.\n\nStep 1. is skipped if nothing is provided as indicators, in which case the change metrics are estimated directly from input data.\n\nKeyword arguments\n\nwidth_ind::Int=100, stride_ind::Int=1: width and stride given to WindowViewer to compute the indicator from the input timeseries.\nwidth_cha::Int=50, stride_cha::Int=1: width and stride given to WindowViewer to compute the change metric timeseries from the indicator timeseries.\nwhichtime = midpoint: The time vector corresponding to the indicators / change metric timeseries is obtained from t in estimate_changes using the keyword whichtime. Options include:\nlast: use the last timepoint of each window\nmidpoint: use the mid timepoint of each time window\nfirst: use first timepoint of each window\nIn fact, the indicators time vector is computed simply via\nt_indicator = windowmap(whichtime, t; width_ind, stride_ind)\nt_change = windowmap(whichtime, t_indicator; width_cha, stride_cha)\nso any other function of the time window may be given to extract the time point itself, such as mean or median.\nT = Float64: Element type of input timeseries to initialize some computations.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SlidingWindowResults","page":"API","title":"TransitionsInTimeseries.SlidingWindowResults","text":"SlidingWindowResults <: ChangesResults\n\nA struct containing the output of estimate_changes used with SlidingWindowConfig. It can be used for further analysis, visualization, or given to significant_transitions.\n\nIt has the following fields that the user may access\n\nx: the input timeseries.\nt: the time vector of the input timeseries.\nx_indicator, the indicator timeseries (matrix with each column one indicator).\nt_indicator, the time vector of the indicator timeseries.\nx_change, the change metric timeseries (matrix with each column one change metric).\nt_change, the time vector of the change metric timeseries.\nconfig::SlidingWindowConfig, what was used for the analysis.\n\n\n\n\n\n","category":"type"},{"location":"api/#Segmented-window","page":"API","title":"Segmented window","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"SegmentedWindowConfig\nSegmentedWindowResults","category":"page"},{"location":"api/#TransitionsInTimeseries.SegmentedWindowConfig","page":"API","title":"TransitionsInTimeseries.SegmentedWindowConfig","text":"SegmentedWindowConfig <: IndicatorsChangeConfig\nSegmentedWindowConfig(indicators, change_metrics, tseg_start, tseg_end; kwargs...)\n\nA configuration that can be given to estimate_changes. It estimates transitions by estimating indicators and changes in user-defined window segments as follows:\n\nFor each segment specified, estimate the corresponding indicator timeseries by sliding a window over the input timeseries (within the window segment).\nFor each segment of the indicator timeseries, estimate a scalar change metric by applying the change metric over the full segment of the indicator timeseries.d\n\ntseg_start, tseg_end are the starts and ends of the window segments (the window segments may overlap, that's okay). indicators, change_metrics are identical as in SlidingWindowConfig.\n\nThe results output corresponding to SlidingWindowConfig is SegmentedWindowResults.\n\nKeyword arguments\n\nwidth_ind::Int=100, stride_ind::Int=1, whichtime = midpoint, T = Float64: keywords identical as in SlidingWindowConfig.\nmin_width_cha::Int=typemax(Int): minimal width required to perform the change metric estimation. If a segment is not sufficiently long, the change metric is NaN.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SegmentedWindowResults","page":"API","title":"TransitionsInTimeseries.SegmentedWindowResults","text":"SegmentedWindowResults <: ChangesResults\n\nA struct containing the output of estimate_changes used with SegmentedWindowConfig. It can be used for further analysis, visualization, or given to significant_transitions.\n\nIt has the following fields that the user may access\n\nx: the input timeseries.\nt: the time vector of the input timeseries.\nx_indicator::Vector{Matrix}, with x_indicator[k] the indicator timeseries (matrix with each column one indicator) of the k-th segment.\nt_indicator::Vector{Vector}, with t_indicator[k] the time vector of the indicator timeseries for the k-th segment.\nx_change::Matrix, the change metric values with x[k, i] the change metric of the i-th indicator for the k-th segment.\nt_change, the time vector of the change metric.\nconfig::SegmentedWindowConfig, what was used for the analysis.\ni1::Vector{Int} indices corresponding to start time of each segment.\ni2::Vector{Int} indices corresponding to end time of each segment.\nprecomp_change_metrics vector containing the precomputed change metrics of each segment.\n\n\n\n\n\n","category":"type"},{"location":"api/#Slope-change","page":"API","title":"Slope change","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"SlopeChangeConfig\nSlopeChangeResults","category":"page"},{"location":"api/#TransitionsInTimeseries.SlopeChangeConfig","page":"API","title":"TransitionsInTimeseries.SlopeChangeConfig","text":"SlopeChangeConfig <: ChangesConfig\nSlopeChangeConfig(; indicator = nothing, kw...)\n\nA configuration that can be given to estimate_changes. It estimates a change of slope in the timeseries by fitting two connected linear segments to the timeseries, returning the results (i.e., the two-linear fits) as SlopeChangeResults.\n\nKeyword arguments\n\nindicator = nothing: if not nothing. Otherwise it should be a function f(x) -> Real. The slope fitting is then done over an indicator of the timeseries, which itself is estimated via a sliding window exactly as in SlidingWindowConfig.\nwidth_ind, stride_ind, whichtime: exactly as in SlidingWindowConfig if indicator is not nothing.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SlopeChangeResults","page":"API","title":"TransitionsInTimeseries.SlopeChangeResults","text":"SlopeChangeResults <: ChangesResults\n\nA struct containing the output of estimate_changes used with SlopeChangeConfig. It can be used for further analysis, visualization, or given to significant_transitions. The only significance type that you can use this with significant_transitions is SlopeChangeSignificance.\n\nIt has the following fields that the user may access:\n\nx: the input timeseries.\nt: the time vector of the input timeseries.\nx_indicator, the indicator timeseries.\nt_indicator, the time vector of the indicator timeseries.\nt_change, the time the slope changes.\nfitparams = a, b, c, d, the fitted linear coefficients, a + b*t before. t_change and c + d*t after t_change.\n\n\n\n\n\n","category":"type"},{"location":"api/#Significance-testing","page":"API","title":"Significance testing","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"significant_transitions\nSignificance\nSurrogatesSignificance\nThresholdSignificance\nSigmaSignificance\nQuantileSignificance\nSlopeChangeSignificance","category":"page"},{"location":"api/#TransitionsInTimeseries.significant_transitions","page":"API","title":"TransitionsInTimeseries.significant_transitions","text":"significant_transitions(res::ChangesResults, signif::Significance)\n\nEstimate significant transtions in res using the method described by signif. Return flags, a Boolean matrix with identical size as the changes stored in res (which typically is stored in the field res.x_change). flags is true wherever a change metric of res is deemed significant.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.Significance","page":"API","title":"TransitionsInTimeseries.Significance","text":"Significance\n\nSupertype used to test for significance in significant_transitions. Changes that are statistically significant are \"transitions\".\n\nValid subtypes are:\n\nSurrogatesSignificance.\nSigmaSignificance.\nQuantileSignificance.\nThresholdSignificance.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SurrogatesSignificance","page":"API","title":"TransitionsInTimeseries.SurrogatesSignificance","text":"SurrogatesSignificance <: Significance\nSurrogatesSignificance(; kwargs...)\n\nA configuration struct for significance testing significant_transitions using timeseries surrogates.\n\nKeyword arguments\n\nsurromethod = RandomFourier(): method to generate surrogates\nn = 1000: how many surrogates to generate\nrng = Random.default_rng(): random number generator for the surrogates\np = 0.05: threshold for significance of the p-value\ntail = :both: tail type used, see below\n\nDescription\n\nWhen used with ChangesResults, significance is estimated as follows: n surrogates from the input timeseries are generated using surromethod, which is any Surrogate subtype provided by TimeseriesSurrogates.jl. For each surrogate, the indicator and then change metric timeseries is extracted. The values of the surrogate change metrics form a distribution of values (one at each time point). The value of the original change metric is compared to that of the surrogate distribution and a p-value is extracted according to the specified tail. The p-value is compared with p to claim significance. After using SurrogatesSignificance, you may access the full p-values before thresholding in the field .pvalues (to e.g., threshold with different p).\n\nThe p-value is simply the proportion of surrogate change metric values that exceed (for tail = :right) or subseed (tail = :left) the original change metric at each given time point. Use tail = :left if the surrogate data are expected to have higher change metric values. This is the case for statistics that quantify entropy. For statistics that quantify autocorrelation, use tail = :right instead. For anything else, use the default tail = :both. An iterable of tail values can also be given, in which case a specific tail is used for each change metric in ChangesResults.\n\nNote that the raw p-values can be accessed in the field .pvalues, after calling the significant_transitions function with SurrogatesSignificance, in case you wish to obtain a different threshold of the p-values.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.ThresholdSignificance","page":"API","title":"TransitionsInTimeseries.ThresholdSignificance","text":"ThresholdSignificance(threshold::Real; tail = :right) <: Significance\n\nA configuration struct for significance testing in significant_transitions. Significance is estimated by comparing the value of each change metric with the given threshold. Values that exceed the threshold (if tail = :right) or subseed the threshold (if tail = :left) are deemed significant. If tail = :both then either condition is checked.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SigmaSignificance","page":"API","title":"TransitionsInTimeseries.SigmaSignificance","text":"SigmaSignificance(; factor = 3.0, tail = :both) <: Significance\n\nA configuration struct for significance testing significant_transitions. When used with ChangesResults, significance is estimated by comparing how many standard deviations (σ) the value exceeds the mean value (μ). Values that exceed (if tail = :right) μ + factor*σ, or subseed (if tail = :left) μ - factor*σ are deemed significant. If tail = :both then either condition is checked.\n\nfactor can also be a vector of values, in which case a different value is used for each change metric.\n\nSee also QuantileSignificance.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.QuantileSignificance","page":"API","title":"TransitionsInTimeseries.QuantileSignificance","text":"QuantileSignificance(; p = 0.95, tail = :both) <: Significance\n\nA configuration struct for significance testing significant_transitions. When used with ChangesResults, significance is estimated by comparing the value of each change metric with its p-quantile. Values that exceed the p-quantile (if tail = :right) or subseed the 1-p-quantile (if tail = :left) are deemed significant. If tail = :both then either condition is checked.\n\nQuantileSignficance guarantees that some values will be significant by the very definition of what a quantile is. See also SigmaSignificance that is similar but does not have this guarantee.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.SlopeChangeSignificance","page":"API","title":"TransitionsInTimeseries.SlopeChangeSignificance","text":"SlopeChangeSignificance(; moe_slope, moe_offset, slope_diff = moe_slope, pvalue = 0.05)\n\nTest whether the result of SlopeChangeResults is statistically significant.\n\nTwo tests are done:\n\nCheck whether the margin of error of the fitted parameters a, b, c, d of the two linear segments a + b*t, c + d*t is less than the specified margins of error, for a chosen pvalue.\nTest that the two slopes b, d have difference greater than slope_diff.\n\nThe Boolean & of the above two is the final test.\n\nThe margin of error is simply half the size of the confidence interval, also known as radius of the confidence interval.\n\n\n\n\n\n","category":"type"},{"location":"api/#indicators","page":"API","title":"Indicators","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Note that any Julia function can be an indicator or change metric, so the list here is only just a couple of indicators directly implemented in this package.","category":"page"},{"location":"api/#Value-distribution","page":"API","title":"Value distribution","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"StatsBase.mean\nStatsBase.skewness\nStatsBase.kurtosis","category":"page"},{"location":"api/#Statistics.mean","page":"API","title":"Statistics.mean","text":"mean(A::AbstractArray, w::AbstractWeights[, dims::Int])\n\nCompute the weighted mean of array A with weight vector w (of type AbstractWeights). If dim is provided, compute the weighted mean along dimension dims.\n\nExamples\n\nn = 20\nx = rand(n)\nw = rand(n)\nmean(x, weights(w))\n\n\n\n\n\n","category":"function"},{"location":"api/#StatsBase.skewness","page":"API","title":"StatsBase.skewness","text":"skewness(v, [wv::AbstractWeights], m=mean(v))\n\nCompute the standardized skewness of a real-valued array v, optionally specifying a weighting vector wv and a center m.\n\n\n\n\n\n","category":"function"},{"location":"api/#StatsBase.kurtosis","page":"API","title":"StatsBase.kurtosis","text":"kurtosis(v, [wv::AbstractWeights], m=mean(v))\n\nCompute the excess kurtosis of a real-valued array v, optionally specifying a weighting vector wv and a center m.\n\n\n\n\n\n","category":"function"},{"location":"api/#Critical-Slowing-Down","page":"API","title":"Critical Slowing Down","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"StatsBase.var\nar1_whitenoise","category":"page"},{"location":"api/#Statistics.var","page":"API","title":"Statistics.var","text":"var(x::AbstractArray, w::AbstractWeights, [dim]; mean=nothing, corrected=false)\n\nCompute the variance of a real-valued array x, optionally over a dimension dim. Observations in x are weighted using weight vector w. The uncorrected (when corrected=false) sample variance is defined as:\n\nfrac1sumw sum_i=1^n w_ileft(x_i - μright)^2 \n\nwhere n is the length of the input and μ is the mean. The unbiased estimate (when corrected=true) of the population variance is computed by replacing frac1sumw with a factor dependent on the type of weights used:\n\nAnalyticWeights: frac1sum w - sum w^2 sum w\nFrequencyWeights: frac1sumw - 1\nProbabilityWeights: fracn(n - 1) sum w where n equals count(!iszero, w)\nWeights: ArgumentError (bias correction not supported)\n\n\n\n\n\nvar(ce::CovarianceEstimator, x::AbstractVector; mean=nothing)\n\nCompute the variance of the vector x using the estimator ce.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.ar1_whitenoise","page":"API","title":"TransitionsInTimeseries.ar1_whitenoise","text":"ar1_whitenoise(x::AbstractVector)\n\nReturn the AR1 regression coefficient of a time series x by computing the analytic solution of the least-square parameter estimation under white-noise assumption for the data-generating process:\n\ntheta = sum_i=2^n x_i x_i-1 sum_i=2^n x_i-1^2\n\n\n\n\n\n","category":"function"},{"location":"api/#Spectrum","page":"API","title":"Spectrum","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"LowfreqPowerSpectrum\nPrecomputedLowfreqPowerSpectrum","category":"page"},{"location":"api/#TransitionsInTimeseries.LowfreqPowerSpectrum","page":"API","title":"TransitionsInTimeseries.LowfreqPowerSpectrum","text":"LowfreqPowerSpectrum(; q_lofreq = 0.1)\n\nReturn a PrecomputableFunction containing all the necessary fields to generate a PrecomputedLowfreqPowerSpectrum. The latter can be initialized by precompute:\n\nlfps = precompute( LowfreqPowerSpectrum() )\n\nKeyword arguments:\n\nq_lofreq: a number between 0 and 1 that characterises which portion of the\n\nfrequency spectrum is considered to be low. For instance, q_lofreq = 0.1 implies that the lowest 10% of frequencies are considered to be the low ones.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.PrecomputedLowfreqPowerSpectrum","page":"API","title":"TransitionsInTimeseries.PrecomputedLowfreqPowerSpectrum","text":"PrecomputedLowfreqPowerSpectrum\n\nA struct containing all the precomputed fields to efficiently perform repetitive computation of the low-frequency power spectrum (LFPS), a number between 0 and 1 that characterizes the amount of power contained in the low frequencies of the power density spectrum of x. Once lfps::PrecomputedLowfreqPowerSpectrum is initialized, it can be used as a function to obtain the LFPS of x::AbstractVector by:\n\nlfps(x)\n\n\n\n\n\n","category":"type"},{"location":"api/#Nonlinear-dynamics","page":"API","title":"Nonlinear dynamics","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Indicators that come from nonlinear timeseries analysis typically quantify some entropy-like or complexity measure from the timeseries. Thousands (literally) such measures are provided out of the box by the ComplexityMeasures.jl package. Given that any of these may be used as an indicator or change metric, we made the decision to not copy-paste any measure here, as it is easy for the user to use any of them.","category":"page"},{"location":"api/","page":"API","title":"API","text":"For example, using the permutation entropy as an indicator is as simple as doing","category":"page"},{"location":"api/","page":"API","title":"API","text":"using ComplexityMeasures\nest = OrdinalPatterns(; m = 3) # order 3\n# create a function that given timeseries returns permutation entropy\nindicator = x -> entropy_normalized(est, x)","category":"page"},{"location":"api/","page":"API","title":"API","text":"and giving the created indicator to e.g., SlidingWindowConfig.","category":"page"},{"location":"api/#change_metrics","page":"API","title":"Change metrics","text":"","category":"section"},{"location":"api/#Slope","page":"API","title":"Slope","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"kendalltau\nspearman\nRidgeRegressionSlope\nPrecomputedRidgeRegressionSlope","category":"page"},{"location":"api/#TransitionsInTimeseries.kendalltau","page":"API","title":"TransitionsInTimeseries.kendalltau","text":"kendalltau(x::AbstractVector)\n\nCompute the kendall-τ correlation coefficient of the time series x. kendalltau can be used as a change metric focused on trend.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.spearman","page":"API","title":"TransitionsInTimeseries.spearman","text":"spearman(x::AbstractVector)\n\nCompute the spearman correlation coefficient of the time series x. spearman can be used as a change metric focused on trend.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.RidgeRegressionSlope","page":"API","title":"TransitionsInTimeseries.RidgeRegressionSlope","text":"RidgeRegressionSlope(; lambda = 0.0) → rr\n\nReturn a PrecomputableFunction containing all the necessary fields to generate a PrecomputedRidgeRegressionSlope. rr can be used as a change metric focused on trend.\n\nlambda is a regularization constant, usually between 0 and 1.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.PrecomputedRidgeRegressionSlope","page":"API","title":"TransitionsInTimeseries.PrecomputedRidgeRegressionSlope","text":"PrecomputedRidgeRegressionSlope\n\nA struct containing the precomputed ridge regression matrix. Once rrslope::PrecomputedRidgeRegressionSlope is initialized, it can be used as a function to obtain the ridge regression slope of x::AbstractVector by applying:\n\nrrslope(x)\n\n\n\n\n\n","category":"type"},{"location":"api/#Value-distribution-differences","page":"API","title":"Value distribution differences","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"difference_of_means","category":"page"},{"location":"api/#TransitionsInTimeseries.difference_of_means","page":"API","title":"TransitionsInTimeseries.difference_of_means","text":"difference_of_means(x::AbstractArray)\n\nReturn the absolute difference of the means of the first and second halfs of x. difference_of_means can be used as a change metric focused on value differences. Creating similar statistical differences using other moments instead of mean is trivial. In fact, the source of difference_of_means is just:\n\n# assumes 1-based indexing\nn = length(x)\nx1 = view(x, 1:n÷2)\nx2 = view(x, (n÷2 + 1):n)\nreturn abs(mean(x1) - mean(x2))\n\ndifference_of_means can also sensibly be used for windows of size 2, in which case the change metric timeseries is the same as the abs.(diff(...)) of the indicator timeseries.\n\n\n\n\n\n","category":"function"},{"location":"api/#own_indicator","page":"API","title":"Make your own indicator/metric!","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"The only difference between what is an \"indicator\" and what is a \"change metric\" is purely conceptual. As far as the code base of TransitionsInTimeseries.jl is concerned, they are both functions f: x::AbstractVector{Real} -> f(x)::Real. As a user you may give any such function for an indicator or change metric.","category":"page"},{"location":"api/","page":"API","title":"API","text":"There are situations where you may optimize such a function based on knowledge of input x type and length, in which case you may use PrecomputableFunction:","category":"page"},{"location":"api/","page":"API","title":"API","text":"PrecomputableFunction\nprecompute","category":"page"},{"location":"api/#TransitionsInTimeseries.PrecomputableFunction","page":"API","title":"TransitionsInTimeseries.PrecomputableFunction","text":"PrecomputableFunction\n\nSupertype of structs containing the necessary field to precompute a ::Function by:\n\nprecompute(f::PrecomputableFunction, t)\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.precompute","page":"API","title":"TransitionsInTimeseries.precompute","text":"precompute(f, t)\n\nPrecompute the function f given a time vector t.\n\n\n\n\n\n","category":"function"},{"location":"api/#Surrogates","page":"API","title":"Surrogates","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"For the surrogate generation, you can use any subtype of Surrogate defined in Timeseriessurrogates.jl.","category":"page"},{"location":"api/#Sliding-windows","page":"API","title":"Sliding windows","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"WindowViewer\nwindowmap\nwindowmap!","category":"page"},{"location":"api/#TransitionsInTimeseries.WindowViewer","page":"API","title":"TransitionsInTimeseries.WindowViewer","text":"WindowViewer(x; width, stride)\n\nInitialize an iterator that generates views over the given timeseries x based on a window with a given width, incrementing the window views with the given stride. You can use this directly with map, such as map(std, WindowViewer(x, ...)) would give you the moving-window-timeseries of the std of x.\n\nIf not given, the keywords width, stride are respectively taken as default_window_width(x) and 1.\n\n\n\n\n\n","category":"type"},{"location":"api/#TransitionsInTimeseries.windowmap","page":"API","title":"TransitionsInTimeseries.windowmap","text":"windowmap(f::Function, x::AbstractVector; kwargs...) → mapped_f\n\nA shortcut for first generating a wv = WindowViewer(x; kwargs...) and then applying mapped_f = map(f, wv). If x is accompanied by a time vector t, you probably also want to call this function with t instead of x and with one of mean, midpoint, midvalue as f to obtain a time vector for the mapped_f output.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.windowmap!","page":"API","title":"TransitionsInTimeseries.windowmap!","text":"windowmap!(f::Function, out, x::AbstractVector; kwargs...)\n\nSame as windowmap, but writes the output in-place in out.\n\n\n\n\n\n","category":"function"},{"location":"api/#Load-data","page":"API","title":"Load data","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"load_linear_vs_doublewell()","category":"page"},{"location":"api/#TransitionsInTimeseries.load_linear_vs_doublewell-Tuple{}","page":"API","title":"TransitionsInTimeseries.load_linear_vs_doublewell","text":"load_linear_vs_doublewell()\n\nLoad prototypical data from a linear and a double-well model to test some indicators.\n\n\n\n\n\n","category":"method"},{"location":"api/#Visualization","page":"API","title":"Visualization","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"plot_indicator_changes\nplot_significance!\nplot_changes_significance","category":"page"},{"location":"api/#TransitionsInTimeseries.plot_indicator_changes","page":"API","title":"TransitionsInTimeseries.plot_indicator_changes","text":"plot_indicator_changes(res) → (fig, axs)\n\nReturn fig::Figure and axs::Matrix{Axis}, on which res::ChangesResults has been visualised.\n\nKeyword arguments:\n\ncolors = default_colors sets the colors of the line plots that are to\n\nbe cycled through. The default correspond to the color scheme of Julia Dynamics.\n\n`indicatornames = defaultindicatorlabel(res), chametricnames =\n\ndefaultchametriclabel(res)sets the labels for the indicators and the change metrics, with the default inferring them from the names ofres.config.indicatorsandres.config.change_metrics`.\n\naccent_linewidth = 3 sets the line width for the original signals (the\n\nsurrogates have linewidth = 1)\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.plot_significance!","page":"API","title":"TransitionsInTimeseries.plot_significance!","text":"plot_significance!(axs, res, signif)\n\nUpdate the axs::Matrix{Axis} of a figure obtained with plot_indicator_changes(res) with the signif::SurrogatesSignificance.\n\nKeyword arguments:\n\nflags = nothing provides the significance flags, for instance obtained by\n\nthresholding the pvalues.\n\nnsurro = 20 sets the number of surrogates to plot.\n\n\n\n\n\n","category":"function"},{"location":"api/#TransitionsInTimeseries.plot_changes_significance","page":"API","title":"TransitionsInTimeseries.plot_changes_significance","text":"plot_changes_significance(res, signif) → (fig, axs)\n\nReturn fig::Figure and axs::Matrix{Axis}, on which res::ChangesResults and signif::SurrogatesSignificance have been visualised. The source code is as simple as:\n\nfig, axs = plot_indicator_changes(res; kwargs...)\nplot_significance!(axs, res, signif; kwargs...)\n\nFor more information, refer to plot_indicator_changes and plot_significance!.\n\n\n\n\n\n","category":"function"},{"location":"api/#Utils","page":"API","title":"Utils","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"default_window_width","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"EditURL = \"do-events.jl\"","category":"page"},{"location":"examples/do-events/#Dansgaard-Oescher-events-and-Critical-Slowing-Down","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"The delta^18O timeseries of the North Greenland Ice Core Project (NGRIP) are, to this date, the best proxy record for the Dansgaard-Oeschger events (DO-events). DO-events are sudden warming episodes of the North Atlantic, reaching 10 degrees of regional warming within 100 years. They happened quasi-periodically over the last glacial cycle due to transitions between strong and weak states of the Atlantic Meridional Overturning Circulation and might be therefore be the most prominent examples of abrupt transitions in the field of climate science. We here propose to hindcast these events by applying the theory of Critical Slowing Down (CSD) on the NGRIP data, which can be found here in its raw format. This analysis has already been done in (Boers, 2018) and we here try to reproduce Figure 2.d-f.","category":"page"},{"location":"examples/do-events/#Preprocessing-NGRIP","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Preprocessing NGRIP","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"Data pre-processing is not part of TransitionsInTimeseries.jl, but a step the user has to do before using the package. To present an example with a complete scientific workflow, we will showcase typical data pre-processing here, that consist of the following steps:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"Load the data, reverse and offset it to have time vector = time before 2000 AD.\nFilter non-unique points in time and sort the data.\nRegrid the data from uneven to even sampling.","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"The time and delta^18O vectors resulting from the i-th preprocessing step are respectively called t_i and x_i. The final step consists in obtaining a residual r, i.e. the fluctuations of the system around the attractor, which, within the CSD theory, is assumed to be tracked. Over this example, it will appear that the convenience of TransitionsInTimeseries.jl leads the bulk of the code to be written for plotting and preprocessing.","category":"page"},{"location":"examples/do-events/#Step-1:","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Step 1:","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"using DelimitedFiles, Downloads\n\nfunction load_ngrip()\n tmp = Base.download(\"https://raw.githubusercontent.com/JuliaDynamics/JuliaDynamics/\"*\n \"master/timeseries/NGRIP.csv\")\n data, labels = readdlm(tmp, header = true)\n return reverse(data[:, 1]) .- 2000, reverse(data[:, 2]) # (time, delta-18-0) vectors\nend\n\nt1, x1 = load_ngrip()","category":"page"},{"location":"examples/do-events/#Step-2","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Step 2","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"uniqueidx(v) = unique(i -> v[i], eachindex(v))\nfunction keep_unique(t, x)\n unique_idx = uniqueidx(t)\n return t[unique_idx], x[unique_idx]\nend\n\nfunction sort_timeseries!(t, x)\n p = sortperm(t)\n permute!(t, p)\n permute!(x, p)\n return nothing\nend\n\nt2, x2 = keep_unique(t1, x1)\nsort_timeseries!(t2, x2)","category":"page"},{"location":"examples/do-events/#Step-3","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Step 3","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"using BSplineKit\n\nfunction regrid2evensampling(t, x, dt)\n itp = BSplineKit.interpolate(t, x, BSplineOrder(4))\n tspan = (ceil(minimum(t)), floor(maximum(t)))\n t_even = collect(tspan[1]:dt:tspan[2])\n x_even = itp.(t_even)\n return t_even, x_even\nend\n\ndt = 5.0 # dt = 5 yr as in (Boers 2018)\nt3, x3 = regrid2evensampling(t2, x2, dt)","category":"page"},{"location":"examples/do-events/#Step-4","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Step 4","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"For the final step we drop the indices:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"using DSP\n\nfunction chebyshev_filter(t, x, fcutoff)\n ii = 10 # Chebyshev filtering requires to prune first points of timeseries.\n responsetype = Highpass(fcutoff, fs = 1/dt)\n designmethod = Chebyshev1(8, 0.05)\n r = filt(digitalfilter(responsetype, designmethod), x)\n xtrend = x - r\n return t[ii:end], x[ii:end], xtrend[ii:end], r[ii:end]\nend\n\nfcutoff = 0.95 * 0.01 # cutoff ≃ 0.01 yr^-1 as in (Boers 2018)\nt, x, xtrend, r = chebyshev_filter(t3, x3, fcutoff)","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"Let's now visualize our data in what will become our main figure. For the segmentation of the DO-events, we rely on the tabulated data from (Rasmussen et al., 2014) (which will soon be available as downloadable):","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"using CairoMakie, Loess\n\nfunction loess_filter(t, x; span = 0.005)\n loessmodel = loess(t, x, span = span)\n xtrend = Loess.predict(loessmodel, t)\n r = x - xtrend\n return t, x, xtrend, r\nend\n\nfunction kyr_xticks(tticks_yr)\n tticks_kyr = [\"$t\" for t in Int.(tticks_yr ./ 1e3)]\n return (tticks_yr, tticks_kyr)\nend\n\nfunction plot_do(traw, xraw, tfilt, xfilt, t, r, t_transitions, xlims, xticks)\n fig = Figure(size = (1600, 1200), fontsize = 24)\n\n # Original timeseries with transition marked by vertical lines\n ax1 = Axis(fig[1, 1], xlabel = L\"Time (kyr) $\\,$\", ylabel = L\"$\\delta^{18}$O (permil)\",\n xaxisposition = :top, xticks = xticks)\n lines!(ax1, traw, xraw, color = (:gray70, 0.5))\n lines!(ax1, tfilt, xfilt, color = :gray10, linewidth = 3)\n vlines!(ax1, t_transitions, color = Cycled(1), linewidth = 3)\n\n # Residual timeseries\n ax2 = Axis(fig[2, 1], ylabel = L\"Residual $\\,$\", xticks = xticks,\n xticksvisible = false, xticklabelsvisible = false)\n lines!(ax2, t, r, color = :gray50, linewidth = 1)\n\n # Axes for variance and AC1 timeseries\n ax3 = Axis(fig[3, 1], ylabel = L\"Variance $\\,$\", xticks = xticks,\n xticksvisible = false, xticklabelsvisible = false)\n ax4 = Axis(fig[4, 1], xlabel = L\"Time (kyr) $\\,$\", ylabel = L\"Lag-1 autocor. $\\,$\",\n xticks = xticks)\n\n axs = [ax1, ax2, ax3, ax4]\n [xlims!(ax, xlims) for ax in axs]\n ylims!(axs[1], (-48, -34))\n rowgap!(fig.layout, 10)\n return fig, axs\nend\n\nxlims = (-60e3, -10e3)\nxticks = kyr_xticks(-60e3:5e3:5e3)\nt_rasmussen = -[-60000, 59440, 58280, 55800, 54220, 49280, 46860, 43340, 41460, 40160,\n 38220, 35480, 33740, 32500, 28900, 27780, 23340, 14692, 11703]\ntloess, _, xloess, rloess = loess_filter(t3, x3) # loess-filtered signal for visualization\nfig, axs = plot_do(t3, x3, tloess, xloess, t, r, t_rasmussen, xlims, xticks)\nfig","category":"page"},{"location":"examples/do-events/#Hindcast-on-NGRIP-data","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Hindcast on NGRIP data","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"As one can see... there is not much to see so far. Residuals are impossible to simply eye-ball and we therefore use TransitionsInTimeseries.jl to study the evolution, measured by the ridge-regression slope of the residual's variance and lag-1 autocorrelation (AC1) over time. In many examples of the literature, including (Boers, 2018), the CSD analysis is performed over segments (sometimes only one) of the timeseries, such that a significance value is obtained for each segment. By using SegmentedWindowConfig, dealing with segments can be easily done in TransitionsInTimeseries.jl and is demonstrated here:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"using TransitionsInTimeseries, StatsBase\nusing Random: Xoshiro\n\nac1(x) = sum(autocor(x, [1])) # AC1 from StatsBase\nindicators = (var, ac1)\nchange_metrics = (RidgeRegressionSlope(), RidgeRegressionSlope())\ntseg_start = t_rasmussen[1:end-1] .+ 200\ntseg_end = t_rasmussen[2:end] .- 200\nconfig = SegmentedWindowConfig(indicators, change_metrics,\n tseg_start, tseg_end; whichtime = last, width_ind = Int(200÷dt),\n min_width_cha = 100) # require >=100 data points to estimate change metric\nresults = estimate_changes(config, r, t)\nsignif = SurrogatesSignificance(n = 1000, tail = [:right, :right], rng = Xoshiro(1995))\nflags = significant_transitions(results, signif)","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"That's it! We can now visualise our results with a generic function that we will re-use later:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"function plot_segment_analysis!(axs, results, signif)\n (; t_indicator, x_indicator) = results\n for k in eachindex(t_indicator) # loop over the segments\n for i in axes(signif.pvalues, 2) # loop over the indicators\n if !isnan(signif.pvalues[k, i]) # plot if segment long enough\n # Plot indicator timeseries and its linear regression\n ti, xi = t_indicator[k], x_indicator[k][:, i]\n lines!(axs[i+2], ti, xi, color = Cycled(1))\n m, p = ridgematrix(ti, 0.0) * xi\n if signif.pvalues[k, i] < 0.05\n lines!(axs[i+2], ti, m .* ti .+ p, color = :gray5, linewidth = 3)\n else\n lines!(axs[i+2], ti, m .* ti .+ p, color = :gray60, linewidth = 3)\n end\n end\n end\n end\nend\nplot_segment_analysis!(axs, results, signif)\nfig","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"In (Boers, 2018), 13/16 and 7/16 true positives are respectively found for the variance and AC1, with 16 referring to the total number of transitions. The timeseries actually includes 18 transition but, in (Boers, 2018), some segments are considered too small to be analysed. In contrast, we here respectively find 9/16 true positives for the variance and 3/16 for AC1. We can track down the discrepancies to be in the surrogate testing, since the indicator timeseries computed here are almost exactly similar to those of (Boers, 2018). This mismatch points out that packages like TransitionsInTimeseries.jl are desirable for research to be reproducible, especially since CSD is gaining attention - not only within the scientific community but also in popular media.","category":"page"},{"location":"examples/do-events/#CSD:-only-a-necessary-condition,-only-in-some-cases","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"CSD: only a necessary condition, only in some cases","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"For codimension-1 systems, approaching a fold, Hopf or transcritical bifurcation implies a widening of the potential U, which defines the deterministic term f = -U of the SDE's right-hand-side. In the presence of noise, this leads to CSD, which is therefore a necessary condition for crossing one of these bifurcations - although it is not always assessable by analysing the timeseries due to practical limitations (e.g. sparse data subject to large measurement noise). It is nonetheless not given that DO-events, as many other real-life applications, can be seen as a codimension-1 fold, Hopf or transcritical bifurcations. Besides this, we emphasise that CSD is not a sufficient condition for assessing a transition being ahead in near future, since a resilience loss can happen without actually crossing any bifurcation. This can be illustrated on the present example by performing the same analysis only until few hundred years before the transition:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"tseg_end = t_rasmussen[2:end] .- 700 # stop analysis 500 years earlier than before\nconfig = SegmentedWindowConfig(indicators, change_metrics,\n tseg_start, tseg_end, whichtime = last, width_ind = Int(200÷dt),\n min_width_cha = 100)\nresults = estimate_changes(config, r, t)\nsignif = SurrogatesSignificance(n = 1000, tail = [:right, :right], rng = Xoshiro(1995))\nflags = significant_transitions(results, signif)\nfig, axs = plot_do(t3, x3, tloess, xloess, t, r, t_rasmussen, xlims, xticks)\nplot_segment_analysis!(axs, results, signif)\nfig","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"For the variance and AC1, we here respectively find 6 and 3 positives, although the transitions are still far ahead. This shows that what CSD captures is a potential widening induced by a shift of the forcing parameter rather than the actual transition. We therefore believe, as already suggested in some studies, that \"resilience-loss indicators\" is a more accurate name than \"early-warning signals\" when using CSD.","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"We draw attention upon the fact that the delta^18O timeseries is noisy and sparsely re-sampled. Furthermore, interpolating over time introduces a potential bias in the statistics, even if performed on a coarse grid. The NGRIP data therefore represents an example that should be handled with care - as many others where CSD analysis has been applied on transitions in the field of geoscience. To contrast with this, we propose to perform the same analysis on synthethic DO data, obtained from an Earth Model of Intermediate Complexity (EMIC).","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"warning: Degrees of freedom\nThese sources of error come along the usual problem of arbitrarily choosing (1) a filtering method, (2) windowing parameters and (3) appropriate metrics (for instance when the forcing noise is suspected to be correlated). This leads to a large number of degrees of freedom (DoF). Although sensible guesses are possible here, checking that results are robust w.r.t. the DoF should be a standard practice.","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"info: Future improvement\nSupporting the computations for uneven timeseries is a planned improvement of TransitionsInTimeseries.jl. This will avoid the need of regridding data on coarse grids and will prevent from introducing any bias.","category":"page"},{"location":"examples/do-events/#Hindcasting-simulated-DO-events","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Hindcasting simulated DO-events","text":"","category":"section"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"In CLIMBER-X, the EMIC described in (Willeit et al., 2022), DO-like events can be triggered by forcing the North Atlantic with a (white noise) freshwater input. Simulated DO-like events present the big advantage of being evenly sampled in time and free of measurement noise. We run this analysis over two exemplary simulation outputs:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"t_transitions = [[1, 1850, 2970, 3970, 5070, 5810, 7050, 8050],\n [1, 3500, 4370, 5790, 7200, 8140]]\nt_lb = [[300, 500, 300, 600, 300, 500, 500], [1800, 500, 1000, 900, 500]]\ntseg_start = [t_transitions[1][1:end-1] + t_lb[1], t_transitions[2][1:end-1] + t_lb[2]]\ntseg_end = [t_transitions[1][2:end] .- 50, t_transitions[2][2:end] .- 50]\n\nfigvec = Figure[]\n\nfor j in 1:2\n # Download the data and perform loess filtering on it\n tmp = Base.download(\"https://raw.githubusercontent.com/JuliaDynamics/JuliaDynamics/\" *\n \"master/timeseries/climberx-do$(j)-omaxa.csv\")\n data = readdlm(tmp)\n tcx, xcx = data[1, 1000:end], data[2, 1000:end]\n t, x, xtrend, r = loess_filter(tcx, xcx, span = 0.02)\n\n # Initialize figure\n xlims = (0, last(tcx))\n xticks = kyr_xticks(xlims[1]:1e3:xlims[2])\n fig, axs = plot_do(tcx, xcx, t, xtrend, t, r, t_transitions[j], extrema(t), xticks)\n ylims!(axs[1], (5, 40))\n axs[1].ylabel = L\"Max. Atlantic overturning (Sv) $\\,$\"\n\n # Run sliding analysis and update figure with results\n dt = mean(diff(tcx))\n config = SegmentedWindowConfig(\n indicators, change_metrics, tseg_start[j], tseg_end[j],\n whichtime = last, width_ind = Int(200÷dt), min_width_cha = 100)\n results = estimate_changes(config, r, t)\n signif = SurrogatesSignificance(n = 1_000, tail = [:right, :right], rng = Xoshiro(1995))\n flags = significant_transitions(results, signif)\n\n plot_segment_analysis!(axs, results, signif)\n vlines!(axs[1], t_transitions[j], color = Cycled(1), linewidth = 3)\n push!(figvec, fig)\nend\nfigvec[1]","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"It here appears that not all transitions are preceeded by a significant increase of variance and AC1, even in the case of clean and evenly sampled time series. Let's check another case:","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"figvec[2]","category":"page"},{"location":"examples/do-events/","page":"Dansgaard-Oescher events and Critical Slowing Down","title":"Dansgaard-Oescher events and Critical Slowing Down","text":"Same here! Although CLIMBER-X does not represent real DO-events, the above-performed analysis might be hinting at the fact that not all DO transitions can be forecasted with CSD. Nonetheless, performing a CSD analysis can inform on the evolution of a system's resilience.","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"EditURL = \"ks_paleojump.jl\"","category":"page"},{"location":"examples/ks_paleojump/#Kolmogorov-Smirnov-test-for-detecting-transitions-in-paleoclimate-timeseries","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"The goal of this example is to show how simple it is to re-create an analysis similar to what was done in the paper \"Automatic detection of abrupt transitions in paleoclimate records\", (Bagniewski et al., 2021). The same analysis was then used to create a database of transitions in paleoclimate records in (Bagniewski et al., 2023) Using TransitionsInTimeseries.jl and HypothesisTests.jl, the analysis becomes a 10-lines-of-code script (for a given timeseries).","category":"page"},{"location":"examples/ks_paleojump/#Scientific-background","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Scientific background","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"The approach of (Bagniewski et al., 2021) is based on the two-sample Kolmogorov Smirnov test. It tests whether the samples from two datasets or timeseries are distributed according to the same cumulative density function or not. This can be estimated by comparing the value of the KS-statistic versus some threshold that depends on the required confidence.","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"The application of this test for identifying transitions in timeseries is simple:","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"A sliding window analysis is performed in the timesries\nIn each window, the KS statistic is estimated between the first half and the second half of the timeseries within this window.\nTransitions are defined by when the KS statistic exceeds a particular value based on some confidence. The transition occurs in the middle of the window.","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"We should point out that in (Bagniewski et al., 2021) the authors did a more detailed analysis: analyzed many different window widths, added a conditional clause to exclude transitions that do not exceed a predefined minimum \"jump\" in the data, and also added another conditional clause that filtered out transitions that are grouped in time (which is a natural consequence of using the Kolmogorov-Smirov test for detecting transitions).","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"Here we won't do that post processing, mainly because it is rather simple to include these additional conditional clauses to filter transitions after they are found.","category":"page"},{"location":"examples/ks_paleojump/#Steps-for-TransitionsInTimeseries.jl","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Steps for TransitionsInTimeseries.jl","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"Doing this kind of work with TransitionsInTimeseries.jl is so easy you won't even trip! This analysis follows the same sliding window approach showcased in our Tutorial, and it even excludes the \"indicator\" aspect: the change metric is estimated directly from the input data!","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"As such, we really only need to define/do these things before we have finished the analysis:","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"Load the input data (we will use the same example as the NGRIP data of Dansgaard-Oescher events and Critical Slowing Down example) and set the appropriate time window.\nDefine the function that estimates the change metric (i.e., the KS-statistic)\nPerform the sliding window analysis as in the Tutorial with estimate_changes\nEstimate the \"confident\" transitions in the data by comparing the estimated KS-statistic with a predefined threshold.","category":"page"},{"location":"examples/ks_paleojump/#Load-timeseries-and-window-length","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Load timeseries and window length","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"Following the Dansgaard-Oescher events example, we load the data after all the processing steps done in that example:","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"using DelimitedFiles, CairoMakie\n\ntmp = Base.download(\"https://raw.githubusercontent.com/JuliaDynamics/JuliaDynamics/\"*\n \"master/timeseries/NGRIP_processed.csv\")\ndata = readdlm(tmp)\nt, xtrend, xresid, xloess = collect.(eachcol(data))\n\nfig, ax = lines(t, xtrend; axis = (ylabel = \"NGRIP (processed)\", xlabel = \"time\"))\nlines!(ax, t, xloess; linewidth = 2)\nfig","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"For the window, since we are using a sliding window here, we will be using a window of length 500 (which is approximately 1/2 to 1/4 the span between typical transitions found by (Rasmussen et al., 2014)).","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"window = 500","category":"page"},{"location":"examples/ks_paleojump/#Defining-the-change-metric-function","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Defining the change metric function","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"HypothesisTest.jl implements the Kolmogorov-Smirnov test, however here we are interested in the value of the test iself (the so-called KS-statistic), rather than a p-value. To this end, we define the following function to compute the statistic, which also normalizes it as in (Bagniewski et al., 2021).","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"using HypothesisTests\n\nfunction normalized_KS_statistic(timeseries)\n N = length(timeseries)\n i = N÷2\n x = view(timeseries, 1:i)\n y = view(timeseries, (i+1):N)\n kstest = ApproximateTwoSampleKSTest(x, y)\n nx = ny = i # length of each timeseries half of total\n n = nx*ny/(nx + ny) # written fully for concreteness\n D_KS = kstest.δ # can be compared directly with sqrt(-log(α/2)/2)\n # Rescale according to eq. (5) of the paper\n rescaled = 1 - ((1 - D_KS)/(1 - sqrt(1/n)))\n return rescaled\nend\n\nN = 1000 # the statistic is independent of `N` for large enough `N`!\nx = randn(N)\ny = 1.8randn(N) .+ 1.0\nz = randn(N)\nw = 0.6randn(N) .- 2.0\n\nfig, ax = density(x; color = (\"black\", 0.5), strokewidth = 4.0, label = \"reference distribution\")\nax.title = \"showcase of normalized KS-statistic\"\nfor q in (y, z, w)\n D_KS = normalized_KS_statistic(vcat(x, q))\n density!(ax, q; label = \"D_KS = $(D_KS)\")\nend\naxislegend(ax)\nfig","category":"page"},{"location":"examples/ks_paleojump/#Perform-the-sliding-window-analysis","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Perform the sliding window analysis","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"This is just a straightforward call to estimate_changes. In fact, it is even simpler than the tutorial. Here we skip completely the \"indicator\" estimation step, and we evaluate the change metric directly on input data. We do this by simply passing nothing as the indicators.","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"using TransitionsInTimeseries\n\nconfig = SlidingWindowConfig(nothing, normalized_KS_statistic; width_cha = 500)\n\nresults = estimate_changes(config, xtrend, t)","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"Which we can visualize","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"function visualize_results(results)\n fig, ax1 = lines(t, xtrend; axis = (ylabel = \"NGRIP (processed)\",))\n ax2, = lines(fig[2, 1], results.t_change, vec(results.x_change), axis = (ylabel = \"D_KS (normalized)\", xlabel = \"time\"))\n linkxaxes!(ax1, ax2)\n hidexdecorations!(ax1; grid = false)\n xloess_normed = (xloess .- minimum(xloess))./(maximum(xloess) - minimum(xloess))\n lines!(ax2, t, xloess_normed; color = (\"gray\", 0.5))\n fig\nend\n\nvisualize_results(results)","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"By overplotting the (smoothened) NGRIP timeseries and the normalized KS-statistic, it already becomes pretty clear that the statistic peaks when transitions occur.","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"The same thing happens if we alter the window duration","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"config = SlidingWindowConfig(nothing, normalized_KS_statistic; width_cha = 200)\nresults = estimate_changes(config, xtrend, t)\nvisualize_results(results)","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"So one can easily obtain extra confidence by varying window size as in (Bagniewski et al., 2021).","category":"page"},{"location":"examples/ks_paleojump/#Identifying-\"confident\"-transitions","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Identifying \"confident\" transitions","text":"","category":"section"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"As this identification here is done via a simple threshold, identifying the transitions is a nearly trivial call to significant_transitions with ThresholdSignificance","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"signif = ThresholdSignificance(0.5) # or any other threshold\nflags = significant_transitions(results, signif)\n\nfig = visualize_results(results)\naxDKS = content(fig[2,1])\nvlines!(axDKS, results.t_change[vec(flags)], color = (\"red\", 0.25))\nfig","category":"page"},{"location":"examples/ks_paleojump/","page":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","title":"Kolmogorov-Smirnov test for detecting transitions in paleoclimate timeseries","text":"We could proceed with a lot of preprocessing as in (Bagniewski et al., 2021) but we skip this here for the sake of simplicity.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"EditURL = \"logistic.jl\"","category":"page"},{"location":"examples/logistic/#Permutation-entropy-for-dynamic-regime-changes","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Permutation entropy is used frequently to detect a transition between one dynamic regime to another. It is useful when the mean and std. of the timeseries values are very similar between the two regimes, which would mean that common distribution-based indicators, or common critical-slowing-down based indicators, would fail.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"This example will also explore different ways to test for significance that are arguably better suitable in such an application than the Tutorial's default of significance via random Fourier surrogates.","category":"page"},{"location":"examples/logistic/#Logistic-map-timeseries","page":"Permutation entropy for dynamic regime changes","title":"Logistic map timeseries","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"A simple example of this is transitions from periodic to weakly chaotic to chaotic motion in the logistic map. First, let's generate a timeseries of the logistic map","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"using DynamicalSystemsBase\nusing CairoMakie\n\n# time-dependent logistic map, so that the `r` parameter increases with time\nr1 = 3.83\nr2 = 3.86\nN = 2000\nrs = range(r1, r2; length = N)\n\nfunction logistic_drifting_rule(u, rs, n)\n r = rs[n+1] # time is `n`, starting from 0\n return SVector(r*u[1]*(1 - u[1]))\nend\n\nds = DeterministicIteratedMap(logistic_drifting_rule, [0.5], rs)\nx = trajectory(ds, N-1)[1][:, 1]","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Plot it, using as time the parameter value (they coincide)","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"fig, ax = lines(rs, x; linewidth = 0.5)\nax.xlabel = \"r (time)\"\nax.ylabel = \"x\"\nfig","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"In this example there is a rather obvious transition to strongly chaotic motion at r ≈ 3.855. However, there is also a subtle transition to weak chaos at r ≈ 3.847. This transition is barely visible in the timeseries, and in fact many of the timeseries statistical properties remain identical.","category":"page"},{"location":"examples/logistic/#Using-a-simpler-change-metric","page":"Permutation entropy for dynamic regime changes","title":"Using a simpler change metric","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Now, let's compute various indicators and their changes, focusing on the permutation entropy as an indicator. We use order 4 here, because we know that to detect changes in a period m we would need an order ≥ m+1 permutation entropy.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"using TransitionsInTimeseries, ComplexityMeasures\n\nfunction permutation_entropy(m)\n est = SymbolicPermutation(; m) # order 3\n indicator = x -> entropy_normalized(est, x)\n return indicator\nend\n\nindicators = (var, ar1_whitenoise, permutation_entropy(4))\nindistrings = (\"var\", \"ar1\", \"pe\")","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"In this example there is no critical slowing down; instead, there is a sharp transition between periodic and chaotic motion. Hence, we shouldn't be using any trend-based change metrics. Instead, we will use the most basic change metric, difference_of_means. With this metric it also makes most sense to use as stride half the window width","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"metric = (difference_of_means, difference_of_means, difference_of_means)\n\nwidth_ind = N÷100\nwidth_cha = 20\nstride_cha = 10\n\nconfig = SlidingWindowConfig(indicators, metric;\n width_ind, width_cha, stride_cha,\n)\n\nresults = estimate_changes(config, x, rs)","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Let's now plot the change metrics of the indicators","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"function plot_change_metrics()\n fig, ax = lines(rs, x; axis = (ylabel = \"input\",), figure = (size = (600, 600),))\n hidexdecorations!(ax; grid = false)\n # plot all change metrics\n for (i, c) in enumerate(eachcol(results.x_change))\n ax, = scatterlines(fig[i+1, 1], results.t_change, c;\n axis = (ylabel = indistrings[i],), label = \"input\"\n )\n if i < 3\n hidexdecorations!(ax; grid = false)\n else\n ax.xlabel = \"r (time)\"\n end\n end\n return fig\nend\n\nfig = plot_change_metrics()","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"We already see the interesting results we expect: the permutation entropy shows a striking change as we go from periodic to weakly chaotic motion at r ≈ 3.847. (Remember: the plotted quantity is how much the indicator changes within a time window. High values mean large changes.)","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Due to its construction, permutation entropy will have a spike for periodic data at the start of the timeseries, so we can safely ignore the spike at r ≈ 3.83.","category":"page"},{"location":"examples/logistic/#Significance-via-random-Fourier-surrogates","page":"Permutation entropy for dynamic regime changes","title":"Significance via random Fourier surrogates","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"One way to test for significance would be via the standard way as in the Tutorial, utilizing surrogate timeseries and SurrogatesSignificance.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Let's do it here for an example, but, we have to be careful. It is crucial that for permutation entropy we use :right as the tail, because it is expected that the surrogates will have higher differences in the permutation entropy timeseries (because, if there is no dynamical change, the permutation entropy will stay the same, while in the surrogates there are always random fluctuations!","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"surromethod = RandomFourier()\n\n# Define a function because we will re-use later\nusing Random: Xoshiro\nfunction overplot_surrogate_significance!(fig, surromethod, color = \"black\")\n\n signif = SurrogatesSignificance(;\n n = 1000, tail = [:both, :both, :right], surromethod, rng = Xoshiro(42),\n )\n flags = significant_transitions(results, signif)\n\n # and also plot the flags with same color\n for (i, indicator) in enumerate(indicators)\n # To make things visually clear, we will also plot some example surrogate\n # timeseries for each indicator and change metric pair\n for _ in 1:10\n s = TimeseriesSurrogates.surrogate(x, surromethod)\n p = windowmap(indicator, s; width = width_ind)\n q = windowmap(metric[i], p; width = width_cha, stride = stride_cha)\n lines!(fig[i+1, 1], results.t_change, q; color = (color, 0.2), linewidth = 1)\n end\n # Plot the flags as vertical dashed lines\n vlines!(fig[i+1, 1], results.t_change[flags[:, i]];\n color = color, linestyle = :dash, linewidth = 3\n )\n end\n # add a title to the figure with how we estimate significance\n content(fig[1, 1]).title = \"surrogates: \"*string(nameof(typeof(surromethod)))\nend\n\nsurromethod = RandomFourier()\noverplot_surrogate_significance!(fig, surromethod)\n\nfig","category":"page"},{"location":"examples/logistic/#Different-surrogates","page":"Permutation entropy for dynamic regime changes","title":"Different surrogates","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Random Fourier surrogates perserve the power spectrum of the timeseries, but the power spectrum is a property integrated over the whole timeseries. It doesn't contain any information highlighting the local dynamics or information that preserves the local changes of dynamical behavior.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"A surrogate type that does a better job in preserving local sharp changes in the timeseries (and hence provides stricter surrogate-based significance) is for example RelativePartialRandomization.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"A much better alternative is to use block-shuffled surrogates, which preserve the short term local temporal correlation in the timeseries and hence also preserve local short term sharp changes in the dynamic behavior.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"surromethod = RelativePartialRandomization(0.25)\nfig = plot_change_metrics()\noverplot_surrogate_significance!(fig, surromethod, \"gray\")\nfig","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Our results have improved. In the permutation entropy, we see only two transitions detected as significant, which is correct: only two real dynamical transitions exist in the data. In the other two indicators we also see fewer transitions, but as we have already discussed, no results with the other indicators should be taken into meaningful consideration, as these indicators are simply inappropriate for what we are looking for here.","category":"page"},{"location":"examples/logistic/#Simpler-Significance","page":"Permutation entropy for dynamic regime changes","title":"Simpler Significance","text":"","category":"section"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"Arguably, exactly because we are using the difference_of_means as a change metric, we may want to be less strict and more simple with our tests for significance. Instead of using SurrogatesSignificance we may use the simpler and much faster SigmaSignificance, which simply claims significant time points whenever a change metric exceeds some pre-defined factor of its timeseries standard deviation.","category":"page"},{"location":"examples/logistic/","page":"Permutation entropy for dynamic regime changes","title":"Permutation entropy for dynamic regime changes","text":"fig = plot_change_metrics()\nflags = significant_transitions(results, SigmaSignificance(factor = 5.0))\n\n# Plot the flags\nfor (i, indicator) in enumerate(indicators)\n vlines!(fig[i+1, 1], results.t_change[flags[:, i]];\n color = Cycled(3), linestyle = :dash, linewidth = 3\n )\nend\ncontent(fig[1, 1]).title = \"significance from std\"\nfig","category":"page"},{"location":"#TransitionsInTimeseries.jl","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"","category":"section"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"(Image: TransitionsInTimeseries.jl)","category":"page"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"TransitionsInTimeseries","category":"page"},{"location":"#TransitionsInTimeseries","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries","text":"TransitionsInTimeseries.jl\n\n(Image: ) (Image: ) (Image: CI) (Image: codecov) (Image: Package Downloads) (Image: DOI)\n\nTransitionsInTimeseries.jl is a free and open-source software to easily analyse transitions within timeseries in a reproducible, performant, extensible and reliable way. In contrast to other existing software with similar target application, TransitionsInTimeseries.jl defines a generic interface for how to find transitions and how to test for significance. Within this interface, it is easy to expand the software in three orthogonal ways:\n\nProvide the analysis pipelines with new indicators, which can be either self-written or imported from other packages. In particular, the latter offers thousands of metrics that can indicate transitions right out of the box.\nAdd new analysis pipelines for finding transitions.\nAdd new ways for significance testing.\n\nTransitionsInTimeseries is a registered Julia package and can be installed by running:\n\n] add TransitionsInTimeseries\n\nAll further information is provided in the documentation, which you can either find online or build locally by running the docs/make.jl file.\n\nAlternative names for this package could have been: Early Warning Signals / Resilience Indicators / Regime-Shift Identifiers / Change-Point Detectors, or however else you want to call them!\n\n\n\n\n\n","category":"module"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"info: Star us on GitHub!\nIf you have found this package useful, please consider starring it on GitHub. This gives us an accurate lower bound of the (satisfied) user count.","category":"page"},{"location":"#content","page":"TransitionsInTimeseries.jl","title":"Content","text":"","category":"section"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"Multi-stable systems can display abrupt transitions between two stability regimes. To predict such transitions in real-world systems solely based on data, mathematical tools have been developed in the last decades. Numerous terminologies have been used for them, such as early warning signals, resilience indicators, regime-shift identifiers, change-point detection and transition indicators. TransitionsInTimeseries.jl sticks to the latter terminology and provides an interface that:","category":"page"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"Allows a fast computation of common transition indicators with a couple of lines, as demonstrated in the example section.\nMakes the surrogate analysis to test for significance under the hub.\nCan be easily extended by any user without touching the source code.\nReduces the programming overhead for any researcher willing to benchmark new methods.\nEases the reproducibility thanks to a clear syntax, a simple installation and RNG-seeded surrogate generation.\nIncreases trustworthiness thanks to a large test suite.","category":"page"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"info: Similar projects\nAn R toolbox and a Python library already exist. However, we believe that they are difficult to extend for the user. Furthermore, they do not offer a native performant code, as here allowed by the use of Julia.","category":"page"},{"location":"#approaches","page":"TransitionsInTimeseries.jl","title":"Approaches","text":"","category":"section"},{"location":"","page":"TransitionsInTimeseries.jl","title":"TransitionsInTimeseries.jl","text":"Over the last decades, research on transition indicators has largely focused on Critical Slowing Down (CSD). CSD is observed when a system approaches a Hopf, a transcritical or a fold bifurcation and consists in a resilience loss of the system. For instance this can be diagnosed by an increase of the variance and the AR1-regression coefficient, as demonstrated in the example section. However, we emphasize that this is one out of many possible approaches for obtaining transition indicators. Recent work has explored new approaches relying on nonlinear dynamics or machine learning. TransitionsInTimeseries.jl is designed to allow these cutting-edge methods and foster the development of new ones.","category":"page"},{"location":"devdocs/#Developer's-documentation","page":"Developer's documentation","title":"Developer's documentation","text":"","category":"section"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"This documentation addresses users that would like to contribute to the software by either solving bugs, improving documentation, or adding new methods. All contributions come in the form of Pull Requests, for which we strongly advise to follow good practices in scientific code, which means that they are properly formatted, documented, tested, etc.","category":"page"},{"location":"devdocs/#New-indicators-or-change-metrics","page":"Developer's documentation","title":"New indicators or change metrics","text":"","category":"section"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"As explained already in e.g., SlidingWindowConfig, new indicators or change metrics are standard Julia functions, so you only need to define such a function (and document it, test it, etc.).","category":"page"},{"location":"devdocs/#New-pipeline-for-estimating-changes","page":"Developer's documentation","title":"New pipeline for estimating changes","text":"","category":"section"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"This means to contribute a fundamentally new \"pipeline\" for estimating/detecting transitions in timeseries. This new pipeline defines what a \"transition\" means. To add a new pipeline follow these steps:","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"Define a new subtype of ChangesConfig\nDefine a new subtype of ChangesResults\nAdd a method for estimate_changes which accepts the new ChangesConfig subtype you defined and returns the ChangesResults subtype you defined.","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"And that's it!","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"Optionally you can further extend:","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"TransitionsInTimeseries.plot_indicator_changes with the new ChangesResults type you defined. Note the plotting functions are in the ext folder of the repository.\nsignificant_transitions(res::ChangesResults, signif::Significance) with your new ChangesResults type and as many Significance subtypes you have the capacity to extend for.","category":"page"},{"location":"devdocs/#New-pipeline-for-estimating-significance","page":"Developer's documentation","title":"New pipeline for estimating significance","text":"","category":"section"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"Statistically significant changes are \"transitions\". However, what \"significant\" means is not universal. There are different ways to test for significance and TransitionsInTimeseries.jl allows various methods.","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"To add a new pipeline for estimating significance follow these steps:","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"Define a new subtype of Significance\nExtend significant_transitions(res::ChangesResults, signif::Significance) with your new type and as many ChangesResults subtypes as you have capacity for.","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"And that's it! Unfortunately so far we have found no way to make the significant_transitions code agnostic of the changes result, so you would need to add a method manually for every ChangesResults.","category":"page"},{"location":"devdocs/","page":"Developer's documentation","title":"Developer's documentation","text":"Optionally you can further extend TransitionsInTimeseries.plot_significance! (which you can find in the ext folder) for visualizing the significance results.","category":"page"},{"location":"refs/#References","page":"References","title":"References","text":"","category":"section"},{"location":"refs/","page":"References","title":"References","text":"Bagniewski, W.; Ghil, M. and Rousseau, D. D. (2021). Automatic detection of abrupt transitions in paleoclimate records. Chaos 31.\n\n\n\nBagniewski, W.; Rousseau, D. D. and Ghil, M. (2023). The PaleoJump database for abrupt transitions in past climates. Scientific Reports 13, 1–18, arXiv:2206.06832.\n\n\n\nBoers, N. (2018). Early-warning signals for Dansgaard-Oeschger events in a high-resolution ice core record. Nature Communications 9, 2556. Accessed on Oct 28, 2022.\n\n\n\nRasmussen, S. O.; Bigler, M.; Blockley, S. P.; Blunier, T.; Buchardt, S. L.; Clausen, H. B.; Cvijanovic, I.; Dahl-Jensen, D.; Johnsen, S. J.; Fischer, H.; Gkinis, V.; Guillevic, M.; Hoek, W. Z.; Lowe, J. J.; Pedro, J. B.; Popp, T.; Seierstad, I. K.; Steffensen, J. P.; Svensson, A. M.; Vallelonga, P.; Vinther, B. M.; Walker, M. J.; Wheatley, J. J. and Winstrup, M. (2014). A stratigraphic framework for abrupt climatic changes during the Last Glacial period based on three synchronized Greenland ice-core records: refining and extending the INTIMATE event stratigraphy. Quaternary Science Reviews 106, 14–28. Accessed on Oct 4, 2023.\n\n\n\nWilleit, M.; Ganopolski, A.; Robinson, A. and Edwards, N. R. (2022). The Earth system model CLIMBER-X v1.0 – Part 1: Climate model description and validation​​​​​​​​​​​​​​. Geoscientific Model Development 15, 5905–5948. Accessed on Aug 11, 2022.\n\n\n\n","category":"page"}] } diff --git a/dev/tutorial/689f0301.png b/dev/tutorial/689f0301.png new file mode 100644 index 0000000..236ee89 Binary files /dev/null and b/dev/tutorial/689f0301.png differ diff --git a/dev/tutorial/73908a15.png b/dev/tutorial/73908a15.png new file mode 100644 index 0000000..32f9d28 Binary files /dev/null and b/dev/tutorial/73908a15.png differ diff --git a/dev/tutorial/8c762359.png b/dev/tutorial/8c762359.png deleted file mode 100644 index 5789437..0000000 Binary files a/dev/tutorial/8c762359.png and /dev/null differ diff --git a/dev/tutorial/d9311396.png b/dev/tutorial/d9311396.png deleted file mode 100644 index cb35ded..0000000 Binary files a/dev/tutorial/d9311396.png and /dev/null differ diff --git a/dev/tutorial/index.html b/dev/tutorial/index.html index 1180c0d..6e7a0d5 100644 --- a/dev/tutorial/index.html +++ b/dev/tutorial/index.html @@ -119,10 +119,10 @@

    We can conveniently plot the information contained in results by using plot_indicator_changes:

    fig = plot_indicator_changes(results)
    Example block output

    Step 2 is to estimate significance using SurrogatesSignificance and the function significant_transitions. Finally, we can conveniently plot the results obtained by updating the figure obtained above with plot_significance!:

    signif = SurrogatesSignificance(n = 1000, tail = [:right, :right])
     flags = significant_transitions(results, signif)
     plot_significance!(fig, results, signif, flags = flags)
    -fig
    Example block output

    Segmented windows

    The analysis shown so far relies on sliding windows of the change metric. This is particularly convenient for transition detection tasks. Segmented windows for the change metric computation are however preferable when it comes to prediction tasks. By only slightly modifying the syntax used so far, one can perform the same computations on segmented windows, as well as visualise the results conveniently:

    config = SegmentedWindowConfig(indicators, change_metrics,
    +fig
    Example block output

    Segmented windows

    The analysis shown so far relies on sliding windows of the change metric. This is particularly convenient for transition detection tasks. Segmented windows for the change metric computation are however preferable when it comes to prediction tasks. By only slightly modifying the syntax used so far, one can perform the same computations on segmented windows, as well as visualise the results conveniently:

    config = SegmentedWindowConfig(indicators, change_metrics,
         t[1:1], t[1200:1200]; whichtime = last, width_ind = 200,
         min_width_cha = 100)
     results = estimate_changes(config, input, t)
     signif = SurrogatesSignificance(n = 1000, tail = [:right, :right])
     flags = significant_transitions(results, signif)
    -fig = plot_changes_significance(results, signif)
    Example block output +fig = plot_changes_significance(results, signif)Example block output