Skip to content

Commit

Permalink
docs: Added resampled M-test and MLL test to the concepts section. Le…
Browse files Browse the repository at this point in the history
…ft Serafini et al., 2024 as in-prep, until pre-print is submitted.
  • Loading branch information
pabloitu committed Nov 26, 2024
1 parent 5f32fbd commit 3b24128
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 3 deletions.
7 changes: 6 additions & 1 deletion docs/concepts/evaluations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ consistency tests and they verify whether a forecast in consistent with an obser
that can be used to compare the performance of two (or more) competing forecasts.
PyCSEP implements the following evaluation routines for grid-based forecasts. These functions are intended to work with
:class:`GriddedForecasts<csep.core.forecasts.GriddedForecast>` and :class:`CSEPCatalogs`<csep.core.catalogs.CSEPCatalog>`.
Visit the :ref:`catalogs reference<catalogs-reference>` and the :ref:`forecasts reference<forecasts-reference>` to learn
Visit the :ref:`catalogs reference<catalogs-reference>` and the :ref:`forecasts reference<forecast-reference>` to learn
more about to import your forecasts and catalogs into PyCSEP.

.. note::
Expand Down Expand Up @@ -105,6 +105,8 @@ Consistency tests
magnitude_test
pseudolikelihood_test
calibration_test
resampled_magnitude_test
MLL_magnitude_test

Publication reference
=====================
Expand All @@ -114,13 +116,16 @@ Publication reference
3. Magnitude test (:ref:`Savran et al., 2020<savran-2020>`)
4. Pseudolikelihood test (:ref:`Savran et al., 2020<savran-2020>`)
5. Calibration test (:ref:`Savran et al., 2020<savran-2020>`)
6. Resampled Magnitude Test (Serafini et al., in-prep)
7. MLL Magnitude Test (Serafini et al., in-prep)

****************************
Preparing evaluation catalog
****************************

The evaluations in PyCSEP do not implicitly filter the observed catalogs or modify the forecast data when called. For most
cases, the observation catalog should be filtered according to:

1. Magnitude range of the forecast
2. Spatial region of the forecast
3. Start and end-time of the forecast
Expand Down
4 changes: 2 additions & 2 deletions docs/getting_started/theory.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1019,7 +1019,7 @@ on the figure by default.
Resampled Magnitude Test
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Aim: Perform the resampled magnitude test for catalog-based forecasts (Serafini et al., 2024),
Aim: Perform the resampled magnitude test for catalog-based forecasts (Serafini et al., , in-prep),
which is a correction to the original M-test implementation that is biased to the total N of
events.

Expand Down Expand Up @@ -1064,7 +1064,7 @@ horizontal line. The quantile score for this forecast is
Modified Multinomial Log-Likelihood (MLL) Magnitude Test
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Aim: Implements the modified Multinomial log-likelihood (MLL) magnitude test (Serafini et al., 2024).
* Aim: Implements the modified Multinomial log-likelihood (MLL) magnitude test (Serafini et al., in-prep).

* Method: Calculates the test statistic distribution as:

Expand Down

0 comments on commit 3b24128

Please sign in to comment.