Updated Inflation Documentation

guide/important-capabilities-dart.rst

@@ -171,31 +171,31 @@ such as the particle filter (PF), please contact the DART team.
 |                    | (KDE) filter               |                                            |
 +--------------------+----------------------------+--------------------------------------------+
 
-DART also has several **inflation algorithms** available for both prior (the
-first value in the namelist) and posterior (the second value in the namelist).
+DART also has several **inflation algorithms** (i.e. "flavors") available for both prior 
+(``inf_flavor``, first value) and posterior (``inf_flavor``, second value) inflation. The
+``inf_flavor`` setting is located under the ``&filter_nml`` heading within ``input.nml``
 The following table lists the inflation “flavors” supported in DART along with
-their type number (set by *inf_flavor* in ``input.nml`` under the “filter_nml”
-section):
-
-+-----------+-----------------------------+----------------------------------+
-| Flavor #  | Inflation flavor name       | References                       |
-+===========+=============================+==================================+
-| 0         | No inflation                | n/a                              |
-+-----------+-----------------------------+----------------------------------+
-| 1         | (Not Supported)             | n/a                              |
-+-----------+-----------------------------+----------------------------------+
-| 2         | Spatially-varying           | **Anderson, J. L.**, 2009. [6]_  |
-|           | state-space (Gaussian)      |                                  |
-+-----------+-----------------------------+----------------------------------+
-| 3         | Spatially-fixed             | **Anderson, J. L.**, 2007. [5]_  |
-|           | state-space (Gaussian)      |                                  |
-+-----------+-----------------------------+----------------------------------+
-| 4         | Relaxation to prior spread  | **Whitaker, J. S.**              |
-|           | (posterior inflation only)  | **and T. M. Hamill**, 2012. [7]_ |
-+-----------+-----------------------------+----------------------------------+
-| 5         | Enhanced spatially-varying  | **El Gharamti M.**, 2018. [8]_   |
-|           | state-space (inverse gamma) |                                  |
-+-----------+-----------------------------+----------------------------------+
+their integer value:
+
++-------------+-----------------------------+----------------------------------+
+| inf_flavor #| Inflation flavor name       | References                       |
++=============+=============================+==================================+
+| 0           | No inflation                | n/a                              |
++-------------+-----------------------------+----------------------------------+
+| 1           | (Not Supported)             | n/a                              |
++-------------+-----------------------------+----------------------------------+
+| 2           | Spatially-varying           | **Anderson, J. L.**, 2009. [6]_  |
+|             | state-space (Gaussian)      |                                  |
++-------------+-----------------------------+----------------------------------+
+| 3           | Spatially-fixed             | **Anderson, J. L.**, 2007. [5]_  |
+|             | state-space (Gaussian)      |                                  |
++-------------+-----------------------------+----------------------------------+
+| 4           | Relaxation to prior spread  | **Whitaker, J. S.**              |
+|             | (posterior inflation only)  | **and T. M. Hamill**, 2012. [7]_ |
++-------------+-----------------------------+----------------------------------+
+| 5           | Enhanced spatially-varying  | **El Gharamti M.**, 2018. [8]_   |
+|             | state-space (inverse gamma) |                                  |
++-------------+-----------------------------+----------------------------------+
 
 DART has the ability to correct for sampling errors in the regression
 caused by finite ensemble sizes. DART’s sampling error correction algorithm

guide/inflation.rst

@@ -16,22 +16,22 @@ departures from the mean) as follows:
  	\overline{\mathbf{x}}_k \right) + \overline{\mathbf{x}}_k, \quad
 	i = 1, 2, ..., N_e`
 
-where :math:`\mathbf{x}_k` is the state at any time :math:`t_k`, :math:`\overline{\mathbf{x}}_k`
+where :math:`\mathbf{x}_k` is the (uninflated) state at any time :math:`t_k`, :math:`\overline{\mathbf{x}}_k`
 is the ensemble mean and :math:`\lambda` is the inflation factor. The inflated ensemble
 members are denoted by :math:`\widetilde{\mathbf{x}}_k^i` and :math:`N_e` is the ensemble size.
-This operation is equivelant to scaling the original variance of the ensemble by a factor of
+This operation is equivalent to scaling the original variance of the ensemble by a factor of
 :math:`\lambda`. Typically, :math:`\lambda > 1` increasing the ensemble variance, yet studies
 such as `El Gharamti (2018) <https://doi.org/10.1175/MWR-D-17-0187.1>`_ have shown that
 :math:`\lambda` could also be less than 1, thereby deflating in the ensemble variance.
 
 Prior or Posterior Inflation?
 -----------------------------
-The EnKF provides two key ensembles: the **prior ensemble** (after the model propagation)
-and the **posterior ensemble** (after the update). **Which ensemble do we inflate?**
+The EnKF provides two key ensembles: the **prior ensemble** (after the model forecast step)
+and the **posterior ensemble** (after the filter update). **Which ensemble do we inflate?**
 
 Historically, inflation was introduced to address sampling errors.
 Recent research, however, demonstrates that prior and posterior inflation can address
-distinct challenges in the filtering problem.
+distinct challenges in the filtering problem including:
 
 * **Prior inflation** tackles deficiencies in the forecast step, such as model biases
 * **Posterior inflation** mitigates issues like sampling and regression errors during the analysis step
@@ -83,21 +83,32 @@ DART includes advanced adaptive inflation algorithms that allow inflation
 to vary dynamically in both space and time. These algorithms treat inflation
 as a random variable characterized by a probability density function (PDF).
 The mean (or mode) of the PDF is used as the inflation value, while the standard
-deviation reflects the level of confidence in that value.
+deviation reflects the level of confidence in that value. In this way, the 
+inflation parameters (mean, standard deviation) are prognostic variables and
+are updated similarly to how model states are updated within the filter step.
 
 * A **larger standard deviation** indicates less confidence in the inflation value, allowing it to evolve more rapidly over time
 * A **smaller standard deviation** signifies greater confidence, leading to slower changes as the inflation value is presumed to be more accurate
 
-.. note::
+.. Important::
 	The adaptive inflation algorithm leverages available observations to update the
 	inflation value. However, if observations suddenly become unavailable
-	(e.g., due to seasonal variations), the algorithm cannot adjust or reduce
-	the inflation. To address this unique case, DART provides an **inflation damping option**.
+	(e.g., due to seasonal variations), the inf_flavor algorithm cannot adjust or reduce
+	the inflation. To address this case, DART provides an **inflation damping option**.
 	This option gradually pushes the inflation value back towards 1, even in the absence
 	of observations. The recommended default value for the damping parameter is **0.9**.
 	However, in cases where inflation needs to be adjusted more aggressively, a lower value,
 	such as **0.6**, can be used.
 
+        It is important to point out that regardless of whether the inflation is updated through
+        the default (inf_flavor) algorithm (observations present) versus the inflation damping
+        option (no observations) **inflation is applied to the ensemble at all times to all variables
+        that are updated during the filter step**.  Thus the ensemble spread continues to change
+        as long as the inflation is not **1**. For circumstances where there are observation gaps
+        (in space or time) the user should take special care to set an appropriate damping parameter
+        to prevent inappropriate ensemble spread that could send the model state into undesirable
+        model space leading to unstable behavior.
+
 The following namelist items which control inflation are found in the ``input.nml`` file,
 in the ``&filter_nml`` namelist. The detailed description is in the
 `filter_mod <../assimilation_code/modules/assimilation/filter_mod.html#Namelist>`_ page. Here we
@@ -107,7 +118,7 @@ Spatial variation is controlled by ``inf_flavor``, which also controls whether t
 by ``inf_sd_initial_from_restart``, ``inf_sd_initial``, ``inf_sd_lower_bound``, ``inf_damping``,
 ``inf_lower_bound``, ``inf_upper_bound`` and ``inf_sd_max_change``.
 
-In the namelist, each entry has two values. The first is for **prior inflation**
+In the namelist, each entry has two values, organized into two columns. The first is for **prior inflation**
 and the second is for **posterior inflation**.
 
 ``&filter_nml :: inf_flavor``
@@ -143,15 +154,17 @@ and the second is for **posterior inflation**.
    or decrease through time and may produce better results.
 
    In practice, we recommend starting with no inflation (both values **0**). Then try
-   inflation type **5** prior inflation and no inflation (**0**) for posterior. Inflation flavor **5** is
-   recommended, over **2** or **3**, because it employs a pdf that is more suitable
-   for describing the inflation unlike the other two flavors as shown in the table above.
+   the inverse gamma inflation type **5** for prior inflation and use no inflation (**0**) 
+   for the posterior. Inflation flavor **5** is recommended, over **2** or **3**, 
+   because it employs a pdf that is more suitable for describing the inflation unlike the 
+   other two flavors as shown in the table above.
 
    We used to support inflation in observation space (flavor **1**) but this has been deprecated.
 
    .. warning::
    	Even if ``inf_flavor`` is not **0**,
-   	inflation will be turned off if ``inf_damping`` is set to **0**.
+   	inflation will be turned off if ``inf_damping`` is set to **0**, which forces the inflation
+        value to be **1**.
 
 Using Relaxation to prior spread (RTPS; flavor **4**):
    RTPS is a *spatially-varying posterior inflation* algorithm. When using RTPS you cannot set the prior inflation
@@ -207,8 +220,10 @@ Using Relaxation to prior spread (RTPS; flavor **4**):
 ``&filter_nml :: inf_damping``
    *valid values:* 0.0 to 1.0
 
-   Applies to all inflation types, but most frequently used with time-adaptive inflation variants. The
-   difference between the current inflation value and 1.0 is multiplied by this factor before the next assimilation
+   Applies to all inflation types, but most frequently used with time-adaptive inflation variants. DART defaults to
+   this damping method to apply inflation updates for regions of state space that are not influenced by observations
+   either because of gaps in availability or are out of range due to localization. To update the inflation, the
+   difference between the current inflation value and 1.0 is multiplied by this ``inf_damping`` value before the next assimilation
    cycle. So the inflation values are pushed towards 1.0, from above or below (if ``inf_lower_bound`` allows inflation
    values less than 1.0). A value of 0.0 turns all inflation off by forcing the inflation value to 1.0. A value of 1.0
    turns damping off by leaving the original inflation value unchanged. We recommend setting the damping to a value
@@ -268,15 +283,31 @@ save the entire series of inflation files to explore how inflation evolves throu
 As part of the workflow, you have to take the output of one assimilation cycle and rename
 it to be the input for the next assimilation cycle. That is the time to make a copy
 that has a unique name - usually with some sort of date or timestamp. This also makes
-it possible to restart an experiment.
-
-.. important::
-		Inflation is only applied to the variables that are **updated** by DART.
-		If you have ``NO_COPY_BACK`` variables in your state, then inflation
-		will not impact them. They are often used to compute the forward
+it possible to restart an experiment. For many of the supported models, the scripting that
+does the renaming and copying of the inflation files may already be provided.
+
+.. Important::
+		Inflation is only applied to the variables that are **updated** by DART
+                by specifying ``UPDATE`` within the ``&model_nml`` section of ``input.nml``.
+		Alternatively, if the variables are not updated (``NO_COPY_BACK``) then inflation
+		will not impact them. ``NO_COPY_BACK`` variables are often used to compute the forward
 		operators, however, they don't take part in the update and as such
 		they will not be inflated.
 
+                Assigning a variable as ``NO_COPY_BACK`` is the **only way** to prevent inflation
+                from being applied to a variable when inflation is turned on.  If a variable is
+                not updated by observations because either 1) observation gaps exist in space or time 
+                2)the state is outside the spatial localization distance, or  3) variable localization 
+                was applied through the ``obs_impact_tool``, --the inflation update for that part of the
+                state will revert to the inflation damping method, and inflation will still be applied 
+                to all parts of the state space.
+
+                DART does not allow for variable specific ``inf_damping`` settings thus if an immediate
+                response time is required in the application of inflation, an alternative is to
+                customize the ``input.nml`` to switch between ``UPDATE`` and ``NO_COPY_BACK`` as needed. 
+                For an example discussing the impact of variable localization on inflation
+                see `issue 777. <https://github.com/NCAR/DART/issues/777>`_
+
 The suggested procedure for testing inflation:
 
 1. Start without any inflation; i.e., ``inf_flavor = 0, 0`` and assess the performance. For a healthy ensemble