Add optional cell to notebooks to reduce spectral resolution

[hugobuddel]

We changed the spectral resolution of the MICADO simulations in #103 . This lead to more accurate results, but now the notebooks take a lot of time and a lot of RAM (30+ GB), to the point that some of them won't run on most peoples computers, and also not on the github test runner.

I prefer to have the simulations accurate by default, even if that means they are slow. But at the same time it must also be easy to run the notebooks, because they are the primary way we communicate with users what to expect from ScopeSim and how to use it.

This PR adds a commented out line to the notebooks with instructions to the user to uncomment that line if they cannot run the simulation. The github test runner will automatically uncomment the line before running the tests (so it must be exactly this line).

The benefit of this PR is twofold:

• Users can run the notebooks.
• The test can run the notebooks.
• The ScopeSim_Data nightly run can fetch all the required data (so ScopeSim can run offline).

It is perhaps OK to require 64GB of RAM (or more?), for the 'real' simulations that people will run on their institute cluster or so, but preferably people should be able to run them locally. We should therefore also figure out where all this memory consumption comes from and address it, but that is not this PR. (I see this PR as a band-aid.)

Note that the tests actually succeed, but I had to trigger them manually, because the notebooks are not ran automatically for a pull request: https://github.com/AstarVienna/irdb/actions/runs/5615876689 . I plan to merge this today, so we can see whether the nightly test run (which do include the notebooks) succeeds.

[oczoske]

I'm not really happy about this but I guess it's acceptable as a band aid. Ideally, some more explanation should be included on what spectral_bin_width is and how it ought to be matched to the resolution of the source spectrum and the dispersion of the MICADO spectra. Personally, I make simulations doable by restricting to a detector window, mostly coincident with the central detector, but the notebooks should probably show the entire array. My laptop cannot do a full array simulation, and I haven't run any of the notebooks to compare the results.

• 4_scopesim_SPEC_pipe_galaxy.ipynb: I don't know what spectra are used here, but it's quite possible that their intrinsic line widths do not even require the full MICADO resolution, hence lowering spectral_bin_width should be fine.
• 6_basic_spectroscopy.ipynb: This uses a sky spectrum from skycalc at a spectral resolution of 5000, which doesn't quite reach what we expect for MICADO. I assume that this is fine as well.
• MICADO_line_lists.ipynb: That's the most tricky one as the lines should be unresolved, i.e. have very small intrinsic line widths. However, for the interpolation in Scopesim to work, the lines have to be presmoothed. For the purpose of this instructional notebook this ought to be explained, and it might be necessary to increase the presmoothing along with spectral_bin_width. I'd been meaning to look at the line_list function, anyway, so I'll keep the notebook in mind.

I don't object to merging for testing whether they run, but the notebooks should be refined a bit more after that.

[oczoske]

MICADO_line_lists.ipynb: You can switch off the relay_psf and micado_ncpas_psf effects, as these should have no effect on a slit-filling source. That will speed up the execution quite a bit and should also reduce memory requirements.

Switching off psfs should be tried first in the linelist thing.

[oczoske]

Okay for the galaxies and sky spectra. For the line list, try switching off PSFs first.

[hugobuddel]

MICADO_line_lists.ipynb: You can switch off the relay_psf and micado_ncpas_psf effects, as these should have no effect on a slit-filling source. That will speed up the execution quite a bit and should also reduce memory requirements.

That is correct, see figures below

• First two figures: The higher resolution simulation looks much better than the low resolution one.
• Last two figures: The inclusion or not of the PSF effects does not affect the simulation (on the low resolution at least).
• I could not create the missing figure: high res and psf effects, as the simulation gets killed at 30GB ram.

High resolution, but no PSF effects included:

Low resolution, also no PSF effects included:

Low resolution, PSF effects are included:

[hugobuddel]

I'm not really happy about this but I guess it's acceptable as a band aid. Ideally, some more explanation should be included on what spectral_bin_width is and how it ought to be matched to the resolution of the source spectrum and the dispersion of the MICADO spectra.

I don't object to merging for testing whether they run, but the notebooks should be refined a bit more after that.

I hit the 'Reference in new issue' item in the three-dot menu on your comment and created #122 . We can discuss it there further.

[hugobuddel]

I'll merge this, and add the sed line also to ScopeSim_Data. Let's see whether everything is green tomorrow.