Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for 3D RM Synthesis --- Outputting NaN for pixels without input data #107

Merged
merged 6 commits into from
Feb 2, 2024

Conversation

jackieykma
Copy link
Contributor

This pull request comes from an action item of the on-going POSSUM Busy Week

It has been identified from the POSSUM 3D pipeline that, for pixels without any valid input data, some of the output data products are set as 0 instead of NaN, which can make it confusing regarding whether those are valid pixels with the output results coincidentally being 0, or that they are actually invalid pixels

This pull request fixes the issue by:
(1) Changing util_misc.toscalar:
Currently, if a fully masked array (e.g., by passing an array full of NaNs through MAD()) is input to toscalar(), the output will be 0.
This can then lead to, for example, the FDF uncertainty being set as 0 instead of NaN (because it can be calculated by toscalar(MAD(...)))

(2) Changing fundamentally how RM-Synthesis is run, through modifying util_RM:
If, for any particular pixel, the Q&U data are all 0 across frequency (e.g., the individual SBID of POSSUM data), then RM-Synthesis will still be run and give FDF=0 across FD.
This has now been fixed by forcing the output FDF to be NaN throughout FD, if the FDF itself is found to be identical to 0 across FD (which result from input Q=U=0)

@jackieykma
Copy link
Contributor Author

I have identified that the changes to toscalar() may have broken some things
I'll look into it and update here

@Cameron-Van-Eck Cameron-Van-Eck merged commit 33bbb43 into CIRADA-Tools:master Feb 2, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants