JP-3121: initial Code to Implement C-Extensions for Ramp Fit

[kmacdonald-stsci]

Resolves JP-3121

The C implementation of ramp fitting is complete, with the exception of the special handling for CHARGELOSS processing of read noise.

All CI tests pass in both STCAL and JWST, with minor changes in JWST CI tests. The JWST tests must be written to enforce type for data arrays. I will open a corresponding JWST PR to account for this.

By default ramp fitting still runs the python code. In ols_fit.py, there is a boolean variable that acts as a switch to run the python code or C code. It is use_c here:

stcal/src/stcal/ramp_fitting/ols_fit.py

Line 662 in 480cec0

use_c = False

If you want to run the C code, set this variable to True.

Checklist

• added entry in CHANGES.rst (either in Bug Fixes or Changes to API)
• updated relevant tests
• updated relevant documentation
• updated relevant milestone(s)
• added relevant label(s)

[zacharyburnett]

The Linux build is failing, but not the MacOS ones:

[kmacdonald-stsci]

The Linux build is failing, but not the MacOS ones:

I'll take a look. Feel free to do as much review as you like. I've basically been the only one to look at this, so would appreciate any eyes.

[kmacdonald-stsci]

Regression test run: https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/1104/

[nden]

I see build errors in the logs. Is the code building correctly?

[codecov]

Codecov Report

Attention: Patch coverage is 46.46018% with 121 lines in your changes missing coverage. Please review.

Project coverage is 83.73%. Comparing base (af5aefb) to head (9a44330).
Report is 48 commits behind head on main.

Files	Patch %	Lines
src/stcal/ramp_fitting/ramp_fit_class.py	7.69%	72 Missing ⚠️
src/stcal/ramp_fitting/ols_fit.py	39.65%	35 Missing ⚠️
tests/test_ramp_fitting.py	86.95%	9 Missing ⚠️
setup.py	0.00%	3 Missing ⚠️
src/stcal/ramp_fitting/ramp_fit.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #156      +/-   ##
==========================================
- Coverage   85.18%   83.73%   -1.46%     
==========================================
  Files          35       35              
  Lines        6797     6990     +193     
==========================================
+ Hits         5790     5853      +63     
- Misses       1007     1137     +130     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[kmacdonald-stsci]

I see build errors in the logs. Is the code building correctly?

I changed the code according to the errors in the log. They didn't cause build problems locally, so didn't know they would cause problems on Jenkins.

I have started a new regression test run and it appears to have built correctly: https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/1105/

[hbushouse]

Is it safe to assume that the CI test "test_downstream / py311-jwst-cov-xdist" errors are due to the fact that the changes in this PR need corresponding changes in jwst in order to run properly?

[hbushouse]

Need to verify whether all issues related to print statements have been resolved. Also, I would suggest leaving the timing calculation in place, at least for now, but turn the reporting of times into debug level log messages, so that we can gather timing info while testing.

[kmacdonald-stsci]

Is it safe to assume that the CI test "test_downstream / py311-jwst-cov-xdist" errors are due to the fact that the changes in this PR need corresponding changes in jwst in order to run properly?

Yes. I have a JP-3121 for JWST, as well. I have a PR open for JWST for this: spacetelescope/jwst#8355

I will rebase and make sure everything for that JWST PR is good to go.

[nden]

🎉 🚀