RCAL-965 Invoke converter from_tvac_raw to enable processing of TVAC/FPS data

[stscieisenhamer]

Resolves RCAL-965

This PR addresses the issue that TVAC/FPS data cannot be run through the pipeline, mostly due to the fact that the TVAC-related datamodels are frozen. The method ScienceRawModel.from_tvac_raw is utilized to ensure that both science and TVAC/FPS data can be processed by the ELP.

The conversion happens in the DQ Init step at the same point that the RampModel consistency is done.

Tasks

• request a review from someone specific, to avoid making the maintainers review every PR
• add a build milestone, i.e. 24Q4_B15 (use the latest build if not sure)
• Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
  • update or add relevant tests
  • update relevant docstrings and / or docs/ page
  • start a regression test and include a link to the running job (click here for instructions)
    • Do truth files need to be updated ("okified")?
      • after the reviewer has approved these changes, run okify_regtests to update the truth files
• if a JIRA ticket exists, make sure it is resolved properly

news fragment change types...

• changes/<PR#>.general.rst: infrastructure or miscellaneous change
• changes/<PR#>.docs.rst
• changes/<PR#>.stpipe.rst
• changes/<PR#>.associations.rst
• changes/<PR#>.scripts.rst
• changes/<PR#>.mosaic_pipeline.rst
• changes/<PR#>.patch_match.rst

steps

• changes/<PR#>.dq_init.rst
• changes/<PR#>.saturation.rst
• changes/<PR#>.refpix.rst
• changes/<PR#>.linearity.rst
• changes/<PR#>.dark_current.rst
• changes/<PR#>.jump_detection.rst
• changes/<PR#>.ramp_fitting.rst
• changes/<PR#>.assign_wcs.rst
• changes/<PR#>.flatfield.rst
• changes/<PR#>.photom.rst
• changes/<PR#>.flux.rst
• changes/<PR#>.source_detection.rst
• changes/<PR#>.tweakreg.rst
• changes/<PR#>.skymatch.rst
• changes/<PR#>.outlier_detection.rst
• changes/<PR#>.resample.rst
• changes/<PR#>.source_catalog.rst

[codecov]

Codecov Report

Attention: Patch coverage is 60.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 78.14%. Comparing base (df41ce2) to head (88a59b0).

Files with missing lines	Patch %	Lines
romancal/dq_init/dq_init_step.py	60.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1596      +/-   ##
==========================================
- Coverage   78.16%   78.14%   -0.02%     
==========================================
  Files         116      116              
  Lines        7643     7647       +4     
==========================================
+ Hits         5974     5976       +2     
- Misses       1669     1671       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[stscieisenhamer]

Initial regression run

[stscieisenhamer]

For all reviewers: I believe all the comments have been addressed. Ready for full review

[braingram]

Regtests with this PR show 1 error in one of the new tests:
https://github.com/spacetelescope/RegressionTests/actions/runs/13316220160
@stscieisenhamer is this expected?

[stscieisenhamer]

Hmmm. So, wrt that fact that the meta is fine, that is what is relevant to this pr.

With respect to the data differences, this runs fine on my mac so it may be a mac vs linux issue, though the differences seem large for that.

So, I'm going with expected and to okayify the results.

[braingram]

LGTM (pending merge of spacetelescope/roman_datamodels#455 and update to the roman_datamodels pin).

[schlafly]

I agree that the amount of duplication between test_wfi_image_pipeline and test_elp_tvac is unfortunate; if one does something horrible like from test_wfi_image_pipeline import a, b, c, d, e, f does that end up doing the right thing? The tests do have value, as you say, but I don't think there's enough motivation to do a significant refactoring to accommodate these weird files and can go along with the copying if there's not an easy alternative.

As Brett says, the regtest is currently failing and so I might avoid merging for B17 unless this is expected.

[braingram]

The CI is in a bad state unrelated to this PR (something is up with the webbpsf data and cache, hopefully #1617 will work around the issue for now).