JP-3525: Update jump and ramp_fitting docs to merge with stcal docs (#…

…8253)

CHANGES.rst

@@ -34,6 +34,9 @@ documentation
 
 - Change docs theme to ``sphinx-rtd-theme`` [#8224]
 
+- Reorganized ``jump`` and ``ramp_fitting`` step docs content that's split between
+  the jwst and stcal repos. [#8253]
+
 emicorr
 -------
 

docs/conf.py

@@ -94,6 +94,7 @@ def check_sphinx_version(expected_version):
     'matplotlib': ('http://matplotlib.org/', None),
     'gwcs': ('https://gwcs.readthedocs.io/en/stable/', None),
     'stdatamodels': ('https://stdatamodels.readthedocs.io/en/latest/', None),
+    'stcal': ('https://stcal.readthedocs.io/en/latest/', None),
 }
 
 if sys.version_info[0] == 2:

docs/jwst/jump/arguments.rst

@@ -1,7 +1,10 @@
+.. _jump_arguments:
+
 Arguments
 =========
 
-The ``jump`` step has 30 optional arguments that can be set by the user:
+The ``jump`` step has many optional arguments that can be set by the user.
+The details for each are listed below.
 
 **Parameters for Baseline Cosmic Ray Jump Detection**
 

docs/jwst/jump/description.rst

@@ -4,101 +4,34 @@ Description
 :Class: `jwst.jump.JumpStep`
 :Alias: jump
 
+This step finds and flags outliers (usually caused by cosmic-ray hits) in
+each pixel of an "up the ramp" IR exposure.
+
 Assumptions
 -----------
-We assume that the ``saturation`` step has already been applied to the input
-science exposure, so that saturated values are appropriately flagged in the
-input GROUPDQ array. We also assume that steps such as the reference pixel
-correction (``refpix``) and non-linearity correction (``linearity``) have been applied, so
-that the input data ramps do not have any non-linearities or noise above the modeled Poisson
+We assume that the :ref:`saturation <saturation_step>` step has already been applied to
+the input exposure, so that saturated ramp groups are appropriately flagged in the
+input GROUPDQ array. We also assume that steps such as
+:ref:`reference pixel correction <refpix_step>` and
+:ref:`non-linearity correction <linearity_step>` have been applied,
+so that the input data ramps do not have any non-linearities or noise above the modeled Poisson
 and read noise due to instrumental effects. The absence of any of these preceding corrections
-or residual non-linearities or noise can lead to the false detection of jumps in the ramps,
-due to departure from linearity.
+or the presence of residual non-linearities and noise can lead to false detection of jumps
+in the ramps, due to departure from linearity.
 
 The ``jump`` step will automatically skip execution if the input data contain fewer
-than 3 groups per integration, because the baseline algorithm requires two first
-differences to work.
-
-Algorithm
----------
-The algorithm for this step is called from the external package ``stcal``, an STScI
-effort to unify common calibration processing algorithms for use by multiple observatories.
-
-This routine detects jumps in an exposure by looking for outliers
-in the up-the-ramp signal for each pixel in each integration within
-an input exposure. On output, the GROUPDQ array is updated with the DQ flag
-"JUMP_DET" to indicate the location of each jump that was found.
-In addition, any pixels that have non-positive or NaN values in the gain
-reference file will have DQ flags "NO_GAIN_VALUE" and "DO_NOT_USE" set in the
-output PIXELDQ array.
-The SCI and ERR arrays of the input data are not modified.
-
-The current implementation uses the two-point difference method described
-in `Anderson & Gordon (2011) <https://ui.adsabs.harvard.edu/abs/2011PASP..123.1237A>`_.
-
-Two-Point Difference Method
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
-The two-point difference method is applied to each integration as follows:
-
-#. Compute the first differences for each pixel (the difference between
-   adjacent groups)
-#. Compute the clipped (dropping the largest difference) median of the first differences for each pixel.
-#. Use the median to estimate the Poisson noise for each group and combine it
-   with the read noise to arrive at an estimate of the total expected noise for
-   each difference.
-#. Compute the "difference ratio" as the difference between the first differences
-   of each group and the median, divided by the expected noise.
-#. If the largest "difference ratio" is greater than the rejection threshold,
-   flag the group corresponding to that ratio as having a jump.
-#. If a jump is found in a given pixel, iterate the above steps with the
-   jump-impacted group excluded, looking for additional lower-level jumps
-   that still exceed the rejection threshold.
-#. Stop iterating on a given pixel when no new jumps are found or only one
-   difference remains.
-#. If the there are only three differences (four groups), the standard median
-   is used rather than the clipped median.
-#. If there are only two differences (three groups), the smallest one is compared to the larger
-   one and if the larger one is above a threshold, it is flagged as a jump.
+than 3 groups per integration, because the baseline algorithm requires at least
+two first differences to work.
 
-#. If flagging of the 4 neighbors is requested, then the 4 adjacent pixels will
-   have ramp jumps flagged in the same group as the central pixel as long as it has
-   a jump between the min and max requested levels for this option.
+Note that the core algorithms for this step are called from the external package
+``stcal``, an STScI effort to unify common calibration processing algorithms
+for use by multiple observatories.
 
-#. If flagging of groups after a ramp jump is requested, then the groups in the
-   requested time since a detected ramp jump will be flagged as ramp jumps if
-   the ramp jump is above the requested threshold.  Two thresholds and times are
-   possible for this option.
-
-Note that any ramp values flagged as SATURATED in the input GROUPDQ array
-are not used in any of the above calculations and hence will never be
-marked as containing a jump.
-
-Multiprocessing
-===============
-This step has the option of running in multiprocessing mode. In that mode it will
-split the input data cube into a number of row slices based on the number of available
-cores on the host computer and the value of the max_cores input parameter. By
-default the step runs on a single processor. At the other extreme if max_cores is
-set to 'all', it will use all available cores (real and virtual). Testing has shown
-a reduction in the elapsed time for the step proportional to the number of real
-cores used. Using the virtual cores also reduces the elapsed time but at a slightly
-lower rate than the real cores.
-
-If multiprocessing is requested the input cube will be divided into a number of
-slices in the row dimension (with the last slice being slightly larger, if needed).
-The slices are then sent to twopoint_difference.py by detect_jumps. After all the
-slices have finished processing, detect_jumps assembles the output group_dq cube
-from the slices.
-
-Subarrays
-=========
-The use of the reference files is flexible. Full-frame reference
-files can be used for all science exposures, in which case subarrays will be
-extracted from the reference file data to match the science exposure, or
-subarray-specific reference files may be used.
+:ref:`Algorithm <stcal:jump_algorithm>`
+---------------------------------------
 
 Large Events (Snowballs and Showers)
-====================================
+------------------------------------
 All the detectors on JWST are affected by large cosmic ray
 events. While these events, in general, affect a large number of
 pixels, the more distinguishing characteristic is that they are
@@ -107,12 +40,40 @@ counts. These excess counts are, in general, below the detection
 threshold of normal cosmic rays.
 
 To constrain the effect of this halo, the jump step will fit ellipses or circles that
-enclose the large events and expand the ellipses and circles by the input expansion_factor
-and mark them as jump.
+enclose the large events and expand the ellipses and circles by the input
+`expand_factor` and mark them as jump (see :ref:`jump step arguments <jump_arguments>`
+for details).
 
-The two types of detectors respond differently. The large events in the near-infrared
+The two different types of JWST detectors respond differently. The large events in the near-infrared
 detectors are almost always circles with a central region that is saturated.
 The saturated core allows the search for smaller events without false positives.
-The MIRI detectors do not, in general, have a saturated center and are only rarely circular.
+The mid-IR (MIRI) detectors do not, in general, have a saturated center and are only rarely circular.
 Thus, we fit the minimum enclosing ellipse and do not require that there are saturated pixels
 within the ellipse.
+
+Multiprocessing
+---------------
+This step has the option of running in multiprocessing mode. In that mode it will
+split the input data cube into a number of row slices based on the number of available
+cores on the host computer and the value of the ``max_cores`` input parameter. By
+default the step runs on a single processor. At the other extreme, if ``max_cores`` is
+set to "all", it will use all available cores (real and virtual). Testing has shown
+a reduction in the elapsed time for the step proportional to the number of real
+cores used. Using the virtual cores also reduces the elapsed time, but at a slightly
+lower rate than the real cores.
+
+If multiprocessing is requested, the input cube will be divided into a number of
+slices in the row dimension (with the last slice being slightly larger, if needed),
+and sent for processing in parallel.
+In the event the number of cores (and hence slices) selected exceeds the number of
+available image rows, the number of slices will be reduced to match the number of rows.
+After all the slices have finished processing, the output GROUPDQ cube - containing
+the DQ flags for detected jumps - is reassembled from the slices.
+
+Subarrays
+---------
+Full-frame reference files can be used for all science exposures even if the
+science exposure was taken in a subarray mode. If so, subarrays will be
+extracted from the reference file data to match the science exposure.
+Alternatively, subarray-specific reference files, which match the science
+exposure, may be used.

docs/jwst/ramp_fitting/appendix.rst

@@ -1,7 +1,12 @@
 Appendix
 ========
 
-The derivation of the segment-specific readnoise variance (:math:`{ var^R_{s}  }`) is shown here. This pertains to both the 'conventional' and 'weighted` readnoise variances - the only difference being the number of groups in the segment.  This derivation follows the standard procedure to fitting data to a straight line, such as in chapter 15 of Numerical Recipes.  The segment-specific variance from read noise corresponds to :math:`{\sigma_b^2}` in section 15.2. 
+The derivation of the segment-specific readnoise variance (:math:`{ var^R_{s}  }`)
+is shown here. This pertains to both the 'conventional' and 'weighted' readnoise
+variances - the only difference being the number of groups in the segment.
+This derivation follows the standard procedure for fitting data to a straight line,
+such as in chapter 15 of Numerical Recipes.  The segment-specific variance from
+read noise corresponds to :math:`{\sigma_b^2}` in section 15.2. 
 
 For read noise R, weight w = :math:`{1 / R^2}`, which is a constant. 
 
@@ -28,7 +33,7 @@ Summations needed:
 :math:`{\sum_{k=1}^n k^2= n \cdot (n+1) \cdot (2 \cdot n+1) / 6 = n^3/3 + n^2/2 +n/6 }`
 
 
-The variance from read noise 
+The variance due to read noise 
 = :math:`{var^R_{s} = S_1 / D = S_1 / (S_1 \cdot S_{xx} - S_x^2)}` 
 
 
@@ -43,4 +48,5 @@ The variance from read noise
 
 = :math:`{ \dfrac{12 \cdot R^2}  {(n^3 - n) \cdot t^2}}` 
 
-This is the equation in the code and in the segment-specific computations section of the Description.
+This is the equation used in the step code and listed in the segment-specific computations
+section of the step Description.

docs/jwst/ramp_fitting/arguments.rst

@@ -1,6 +1,6 @@
 Arguments
 =========
-The ramp fitting step has three optional arguments that can be set by the user:
+The ramp fitting step has the following optional arguments that can be set by the user:
 
 * ``--save_opt``: A True/False value that specifies whether to write
   the optional output product. Default is False.
@@ -17,13 +17,13 @@ The ramp fitting step has three optional arguments that can be set by the user:
   saturated.
 
 * ``--maximum_cores``: The number of available cores that will be
-  used for multi-processing in this step. The default value is '1' which does not use
-  multi-processing. The other options are either an integer, 'quarter', 'half', and 'all'.
+  used for multi-processing in this step. The default value is '1', which results in no
+  multi-processing. Other options are either an integer, 'quarter', 'half', and 'all'.
   Note that these fractions refer to the total available cores and on most CPUs these include
   physical and virtual cores. The clock time for the step is reduced almost linearly by the
   number of physical cores used on all machines. For example, on an Intel CPU with
   six real cores and six virtual cores, setting maximum_cores to 'half' results in a
   decrease of a factor of six in the clock time for the step to run. Depending on the system,
-  the clock time can also decrease even more with maximum_cores is set to 'all'.
+  the clock time can also decrease even more with maximum_cores set to 'all'.
   Setting the number of cores to an integer can be useful when running on machines with a
   large number of cores where the user is limited in how many cores they can use.