Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flux-calibration and resolution matrices #2419

Open
segasai opened this issue Nov 29, 2024 · 1 comment
Open

Flux-calibration and resolution matrices #2419

segasai opened this issue Nov 29, 2024 · 1 comment

Comments

@segasai
Copy link
Contributor

segasai commented Nov 29, 2024

Hi,

It is my understanding that while at extraction time it is true that the extracted spectrum S probably can be written as
$S = R * M$ where M is the "unconvolved" spectrum, and R is the resolution matrix. The unconvolved spectrum however is the uncalibrated spectrum.
Therefore, strictly speaking, after dividing your uncalibrated spectra by the flux calibration vector, the resolution matrix must be updated to still apply.

To first order I think if your the calibration vector is C you need a new resolution matrix which is
$C_d^{-1} R C_d$
where $C_d$ is the diagonal matrix from the calibration vector.
Obviously if calibration vector doesn't change with wavelength ( or is very smooth) , it's a non-issue, but if does have sharp changes (which it does for DESI), then the effect will be noticeable.
From my quick look at the code, I don't think this correction is applied in desispec (please correct me if I am wrong).

A quick test on a single z-band exposure shows that ~ 1% of pixels the resolution matrix values change by more than 0.05 (i.e 10% of the peak value).

From what I can see the change is kind'a trivial -- just update this :

def apply_flux_calibration(frame, fluxcalib):

by changing the resolution matrix after calibration.

Thoughts ?

Thanks,
S

PS I haven't tested this obviously, but if that is a correct thing to do I'd expect this to lead to improvement in chi-squares.

@segasai
Copy link
Contributor Author

segasai commented Dec 3, 2024

With the #2422 merged, I intend to proceed to make a patch for this issue, as I think this is the right approach.
The only tricky thing I can see is what to do if the calibration vector is invalid somewhere (==0), hence you can't invert the matrix that I specified.
Here is my thinking for the correct procedure.

If we have and extracted spectrum S and some kind of true model spectrum M and resolution matrix R, then
$E[S] = R M$

If we now want to use the calibrated spectrum $S_c = C_d^{-1} S$ where ($C_d$ is the diagonal matrix made from calibration vector), and calibrated model spectrum $M_c = C_d^{-1} M$,
then for $E[S_c] = {\tilde R} M_c$ we need ${\tilde R} = C_d^{-1} R C_d$

Now the question is, what if when we calibrate some of the $C_d$ are zeros (i.e. we can't calibrate some pixels). I.e. instead of $C_d^{-1}$ to calibrate we use $C_{d,inv}$ which has zeros at those bad locations.
I argue the right way to thing about it is:

Imagine we can come up with with the calibration vector where all zeros in $C_d$ are patched up, i.e interpolated away for example. Lets call this $C_f$.
Then what we are after is the resolution matrix $R_1$ such that
$E[ S_c ] = R_1 M_c$
where $S_c = C_{d,inv} S$
and $M_c= C_f^{-1} M$
I.e. what we are after the resolution matrix that brings the calibrated model spectrum into what we observe (but that calibrated model spectrum is calibrated with patched up fluxcalib)

If we agree on what's above therefore
$E[C_{d,inv} S] = R_1 C_f^{-1} M$
$C_{d,inv} R M = R_1 C_f^{-1} M$
And then $R_1 = C_{d,inv} R C_f$

Therefore it means, when we apply the $C_{d}^{-1} R C_d$ equation. if some of the C_d are zero, we use pseudo inverse on the left, and we use interpolated C_d on the right.

Any thoughts ?

If it makes sense, I'll write a patch for apply_fluxcalibration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant