Skip to content

Commit

Permalink
Decrease RAM usage of RMtools (#109)
Browse files Browse the repository at this point in the history
* Decrease RAM usage of RMtools

This decreases the peak RAM usage by about 40-50 percent when write_separate_RMSF=True as we don't need to keep everything in memory.
Additionally, it creates the option to do the RM synthesis and RMSF calculation separately instead of in the same function which decreases the RAM usage of the POSSUM Polarimetry pipeline by about 20-25 percent throughout, while not losing any noticable CPU time.
To make this change, I did have to make sure lambda0Sq_m2 is reported by util_RM.py and handled correctly by both do_RMsynth_1D and do_RMsynth_3D.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Alec Thomson (S&A, Kensington WA) <[email protected]>
Co-authored-by: Cameron Van Eck <[email protected]>
  • Loading branch information
ErikOsinga authored Feb 18, 2024
1 parent a9ef125 commit 12e821d
Show file tree
Hide file tree
Showing 6 changed files with 276 additions and 215 deletions.
3 changes: 1 addition & 2 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,5 +37,4 @@ jobs:
- name: Test with pytest
shell: bash -l {0}
run: |
cd tests/
pytest *.py
pytest
2 changes: 1 addition & 1 deletion RMtools_1D/do_RMsynth_1D.py
Original file line number Diff line number Diff line change
Expand Up @@ -324,7 +324,7 @@ def run_rmsynth(
)

# Calculate the Rotation Measure Spread Function
RMSFArr, phi2Arr_radm2, fwhmRMSFArr, fitStatArr = get_rmsf_planes(
RMSFArr, phi2Arr_radm2, fwhmRMSFArr, fitStatArr, _ = get_rmsf_planes(
lambdaSqArr_m2=lambdaSqArr_m2,
phiArr_radm2=phiArr_radm2,
weightArr=weightArr,
Expand Down
Loading

0 comments on commit 12e821d

Please sign in to comment.