1.11.1 jump

[stscijgbot-jp]

Issue JP-3286 was created on JIRA by Misty Cracraft:

jump

Updated the code to handle the switch to sigma clipping for exposures with at least 101 integrations. Three new parameters were added to the jump step to control this process.

Also, updated the code to enter the values for the cosmic ray rate and the snowball/shower rate into the FITS header. [#⁠7609, spacetelescope/stcal#⁠174]

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

Tested a file through default parameters of the jump file (does not call shower code) and see these two header keywords in the output rate file:

PRIMECRS=     0.58808322092086 / Cosmic Ray rate per thousand pixels per second 
EXTNCRS =                  0.0 / Snowballs/showers per million pixels per second

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

That's the expected result when using default params, which only does "normal" CR detection. If you rerun with extended flagging turned on, you should see a non-zero value for EXTNCRS too.

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

Yes, I'm running that test now. This was just confirmation that the new keywords are showing up in rate files with the new pipeline version. I'll add another version when I run with the shower code.

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

Well, now I'm wondering about the population of the EXTNCRS header keyword. I ran the shower code on a MASK1140 image, one of those coronagraph images that have shown almost the entire image being wiped out by NaNs when the shower code is run, and still see 0 for that keyword.

FILENAME= 'jw01194015001_04101_00004_mirimagesh_rate.fits' / Name of the file

PRIMECRS=    3.575783339873856 / Cosmic Ray rate per thousand pixels per second 
EXTNCRS =                  0.0 / Snowballs/showers per million pixels per second

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

Also, since this was a patch updated about two business days before the end of the testing deadline, I will not be able to fully test it by July 5th. The MIRI team will need more time to test the patch updates.

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

I am testing with a file from program 2508, the TSO program that triggered a ticket about flagging of pixels in the central image of the trace, and now, whether or not I turn on the shower code, there are no NaNs (or at least no more than a random pixel here and there) in the central portion of the combined rate image in version 1.11.1. So I'm not entirely sure how to say that this update is working. Mike Regan's testing might be more comprehensive here, in what he tested before the patch was included in the release. I can see changes in the values whether or not I have the shower code on, but the combined rate files are largely clean of NaNs in the center of the trace either way. That NaNs that do exist are persistent in both versions.

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

Regarding the population of the EXTNCRS keyword, I just ran a random NIRCam dataset through the jump step with expand_large_events turned on and it too came out with EXTNCRS=0.0. Now it could just be coincidence that the image I picked doesn't have any viable extended events in it. Michael Regan do you have test data in which the EXTNCRS keyword does successfully get populated with a non-zero value?

[stscijgbot-jp]

Comment by Michael Regan on JIRA:

Yes, I have tested in extensively using MIRI data. Checking the NIR source code I see the problem. It doesn't count the Snowballs for non-multiprocessing mode. 
Fix is simple but doesn't seem critical. Regular CRs are counted in non-multiprocessing mode.

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

Michael Regan , do you have coronagraphic data in your MIRI tests? I've attached a screen grab of the file I posted about, with EXTNCRS = 0.0 in it as well, when there are clearly many pixels flagged in the shower code. Do you expect the keyword to be 0 with that many NaNs in the rate image?

[stscijgbot-jp]

Comment by Jo Taylor on JIRA:

NIRISS performed some tests for this, see JP-3243 and JP-3096. We found some potential issues but were unable to test further due to fixes being implemented too close to the build testing deadline

[stscijgbot-jp]

Comment by Alicia Canipe on JIRA:

Closing this ticket now that this round of testing has ended. 

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

The testing that MIRI was able to do showed no major problems, but we were not able to test in-depth due to the fixes being implemented too close to the build testing deadline.