Improve Background Spectra Combination for NIRSpec MOS Master Background

[stscijgbot-jp]

Issue JP-3746 was created on JIRA by Christian Hayes:

We would like to add an option for median combination of background spectra (or another method that handles NaNs and outliers) in the MOS master background step.

 

Details:

In the current spec2 pipeline's NIRSpec MOS master background step, the 1D spectra extracted from all background slits are equally weighted in the combination that creates the 1D master background spectrum. Currently this can introduce bad pixels/missing CRs/etc in the final 1D master background spectrum (which has been reported by users) and can include contaminated backgrounds.  The attached image (mos_master_background_nan_badpix.png) shows an example of an automatic MOS master background spectrum (both the 1D spectrum and one of the projected 2D spectra), which shows an example of a bad pixel (at just over 3 um) and NaNs (at just under 3 um) have been propagated from individual background spectra into the MOS master background spectrum, and will be propagated to all science spectra in the exposure

Currently, the MOS master background step calls functions from combine_1d whose weight options are not exposed to the users and provide identical weights for MOS master background spectra (the weighting options are "integration_time," "exposure_time," and "unit_weight", all of which would be the identical for background slitlets in the same exposure).  We would like to have alternative options (exposed to the users) for combining spectra, preferably some that include a manner of outlier rejection, such as a median or variance-weighted combination of the input spectra.  Particularly a median combination may remove most cases of bad or missing pixels and improve the default processing for this step. 

For example the two plots mos_mean_background.png and mos_median_background.png show examples of MOS background spectra and compare a mean or median 1D spectrum (with NaN handling), showing that in addition to rejecting bad pixels, a median combination will also in principle filter out bad background spectra (e.g., spectra that are contaminated or may come from partially stuck closed shutters).

[stscijgbot-jp]

Comment by David Law on JIRA:

Assigning this to NIRSpec while we sort out the best approach.  Christian Hayes I think there are some common themes between this and potential MRS background improvements too that we can discuss.

[stscijgbot-jp]

Comment by Christian Hayes on JIRA:

David Law who would be best to talk to about the MRS background improvements that MIRI wants?