Script results are not reproducible

[dkirkby]

Running the following command twice gives similar but not identical results, which I wasn't expecting:

quickspecsim -c desi --airmass 1 --exposure-time 48hour --ab-mag 25.3 --filter sdss2010-i

For example:

Median SNR in b camera = 0.898 / 0.5 Angstrom
Median SNR in r camera = 0.712 / 0.5 Angstrom
Median SNR in z camera = 0.509 / 0.5 Angstrom

and

Median SNR in b camera = 0.910 / 0.5 Angstrom
Median SNR in r camera = 0.712 / 0.5 Angstrom
Median SNR in z camera = 0.492 / 0.5 Angstrom

@julienguy Is this perhaps a "feature" of the new fastsim method calculating fiberloss? Do you use any random numbers?

[julienguy]

Yes. lines 710 and 711 of instrument.py, random offsets. You would need to set a random seed
to get reproducible results.

[alxogm]

Hi, @dkirkby @julienguy we face to this issue, or a related one, by looking at the reproducibility of quasar spectra with DLAs features. See desihub/desisim#386. Attempting to solve such issue I made a branch of specsim: test_repr_qquasar which modifies only the generate_random_noise function in simulator.py. What tests do you suggest to make to see if this affects other applications? or to corroborate such modifications makes sense? Thanks

[dkirkby]

The generate_random_noise function should give reproducible results if you pass in a np.random.RandomState() object.

This issue refers to random centroid offsets that are currently not reproducible. You can test if this is the source of the problem you are seeing by setting instrument.offset.sigma1d to zero in your specsim config.