Reference image can't be found if no Direct in same filter as Grism for NIRISS (possibly other instruments too)

[TheSkyentist]

I am working on a NIRISS program with F200W direct and GR150R grism in F200W, F150W, and F115W.

During grism_prep, the code will fail if there are no Direct observations in the same filter as the Grism observations:

This is due to the following:

1. When loading the GroupFLT the following code snippet is run:

https://github.com/gbrammer/grizli/blob/3b3f09da271800163443c0d09091a35a1aa6d81f/grizli/pipeline/auto_script.py#L3071C1-L3084C40

which checks if there is an observation matching both the grism and pupil information. For my program, this will be true for F200W, F150W and F115W.

2. Then, the reference filter is decided based purely on the previous information:

https://github.com/gbrammer/grizli/blob/3b3f09da271800163443c0d09091a35a1aa6d81f/grizli/pipeline/auto_script.py#L3071C1-L3084C40

The result is that the grism looks for a detection image in each of my grism bands, which causes an error:

https://github.com/gbrammer/grizli/blob/3b3f09da271800163443c0d09091a35a1aa6d81f/grizli/pipeline/auto_script.py#L3148C36-L3148C36

The preferred behavior would be the following:

1. Use the matching direct filter for the grism if it exists.
2. If it does not, specify the priority of direct images to use for grism-filter pairs.
3. At the very least, the order of direct filters should be used from the Grism Reference Filters dictionary.

This is especially relevant for pure parallel data where the grism/direct filter combos can be a grab bag.
I don't believe this affects NIRCam after taking a cursory look, but it may have a similar affect.

[gbrammer]

I think I'd prefer that the user have the responsibility for assigning the direct image reference for these cases.

You could do this either by simply making a symbolic link of the file with the expected filename to some other one that actually exists. Or, alternatively you can explicitly specify the direct image reference with the force_ref keyword in pipeline.auto_script.grism_prep:

grizli/grizli/pipeline/auto_script.py

Lines 3296 to 3324 in 7450014

	def grism_prep(field_root='j142724+334246', PREP_PATH='../Prep', EXTRACT_PATH='../Extractions', ds9=None, refine_niter=3, gris_ref_filters=GRIS_REF_FILTERS, force_ref=None, files=None, split_by_grism=True, refine_poly_order=1, refine_fcontam=0.5, cpu_count=0, mask_mosaic_edges=False, prelim_mag_limit=25, refine_mag_limits=[18, 24], init_coeffs=[1.1, -0.5], grisms_to_process=None, pad=(64, 256), model_kwargs={'compute_size': True}, sep_background_kwargs=None, subtract_median_filter=False, median_filter_size=71, median_filter_central=10, second_pass_filtering=False, box_filter_sn=3, box_filter_width=3, median_mask_sn_threshold=None, median_mask_dilate=8, prelim_model_for_median=False, use_jwst_crds=False):
	"""
	Contamination model pipeline for grism exposures
	Parameters
	----------
	field_root : str
	Rootname of the associated data, used to define the direct image, catalog
	filenames
	PREP_PATH : str
	Relative path to the "Prep" directory containing the grism exposures
	EXTRACT_PATH : str
	Relative path to the "Extractions" directory where the outputs will be copied
	ds9 : `~grizli.ds9.DS9`, None
	If an open `DS9` window, will display contamination model as it's calcluated
	refine_niter : int
	Number of refinement iterations with the polynomial continuum models
	gris_ref_filters : dict
	Associated reference images for particular grism elements
	force_ref : str, None
	Explicitly specify the direct image reference. Otherwise will be calculated
	with ``field_root``, ``gris_ref_filters``.

[TheSkyentist]

I agree that the user should specify what the fallback is. However, I thought that was the role Grism Reference Filters dictionary. If that's not the case, perhaps we can patch in a more appropriate error warning statement?

[TheSkyentist]

I've opened #210 to at least add a message to the user if the corresponding direct image can't be found. I'm curious as to why the Grism Reference Filters dictionary isn't used for deciding filters with JWST in the same way as it is with HST. Can you provide any insight into this?