Add end-to-end, SDP-lite regtest

[schlafly]

In order to check out the full SDP interface, we could make a new regtest / smoketest that runs all of the ELP / HLP steps.

This is annoying in that it would be slow. We would want to do something like the following:

• take two overlapping uncal files (e.g., existing 0001 and 0002 imaging files) for WFI01
• run the ELP on both through source catalogs (i.e., skipping tweakreg, as the SDP does), creating "2a" files that aren't archived
• make an association file for the "2a" files
• run tweakreg / the ELP on the "2a" files to produce the final cal files
• make an association file for the cal files for a particular skycell
• run the mosaic pipeline on the association

It's not obvious to me that I would want to do very much with the resulting files (e.g., compare_asdf, etc.); I might continue to leave that to the existing set of regtests. But it's at least important to know that all of those steps run and to have a place with a SDP-lite pipeline that the real SDP pipeline can emulate. This would make us more aware when the SDP commands need to change slightly to accommodate changes to the pipeline.

[schlafly]

While checking a new version of the release I did the following as a kind of SDP-lite set of invocations:

# make level "2a" files
strun roman_elp r0000101001001001001_01101_0001_WFI01_uncal.asdf --steps.tweakreg.skip=True --suffix=sourcedetection
strun roman_elp r0000101001001001001_01101_0002_WFI01_uncal.asdf --steps.tweakreg.skip=True --suffix=sourcedetection
# make association file linking these
asn_from_list *_WFI01_sourcedetection.asdf --product-name aoeu -o asn_sourcedetection.json
# make final cal files
strun romancal.step.TweakRegStep asn_sourcedetection.json --suffix=cal
# make association file for cal files
asn_from_list *_cal.asdf --product-name=r0000101001001001001_F158_visit_r274dp63x32y80 -o mosaic.json
# this file then needs to get hand edited to add the target skycell name to the "target" bit of the asn file
# i.e., add target="r274dp63x32y80" to the json file
# make final mosaic product
strun roman_mos mosaic.json

This makes the following files:

asn_sourcedetection.json  # association file we made for source detection files, could be ignored?
mosaic.json  # association file we made for cal files, could be ignored?
r0000101001001001001_01101_0001_WFI01_cal.asdf  # L2 file
r0000101001001001001_01101_0001_WFI01_cat.asdf  # corresponding cat file
r0000101001001001001_01101_0001_WFI01_segm.asdf  # corresponding segmentation file
r0000101001001001001_01101_0001_WFI01_sourcedetection.asdf  # "2a" file that we shouldn't save
r0000101001001001001_01101_0001_WFI01_uncal.asdf  # uncal input file
r0000101001001001001_01101_0002_WFI01_cal.asdf  # L2 file
r0000101001001001001_01101_0002_WFI01_cat.asdf  # corresponding cat file
r0000101001001001001_01101_0002_WFI01_segm.asdf  # corresponding segm file
r0000101001001001001_01101_0002_WFI01_sourcedetection.asdf  # corresponding sourcedetection file
r0000101001001001001_01101_0002_WFI01_uncal.asdf  # uncal input file
r0000101001001001001_F158_visit_r274dp63x32y80_cat.asdf  # L3 catalog
r0000101001001001001_F158_visit_r274dp63x32y80_i2d.asdf  # L3 image
r0000101001001001001_F158_visit_r274dp63x32y80_segm.asdf  # L3 segmentation map
skycell_wcs.asdf  # temporarry file that should be deleted

I think something of that form would exercise much of the needed SDP interface, but @ctbheller , let me know if there's something else I can add that would be helpful.

[schlafly]

Updating file names and adding the --target argument:

# make level "2a" files
strun roman_elp r0000101001001001001_0001_wfi01_uncal.asdf --steps.tweakreg.skip=True --suffix=sourcedetection
strun roman_elp r0000101001001001001_0001_wfi02_uncal.asdf --steps.tweakreg.skip=True --suffix=sourcedetection
# make association file linking these
asn_from_list *_sourcedetection.asdf --product-name aoeu -o asn_sourcedetection.json
# make final cal files
strun romancal.step.TweakRegStep asn_sourcedetection.json --suffix=cal
# make association file for cal files
asn_from_list *_cal.asdf --product-name=r0000101001001001001_F158_visit_r274dp63x32y80 -o mosaic.json --target r274dp63x32y80
# make final mosaic product
strun roman_mos mosaic.json

works for the proposed intermediate release.