new MICADO data files

[oczoske]

Ric Davies sent new data files for MICADO; these are incorporated into irdb/MICADO in this pull request. Comments on changes made to individual files:

• TER_entrance_window.dat: added some meta keywords; added columns reflection and emissivity = 1 - transmission.
• TER_coating_antireflection.dat: this is a new file that is not used by any effect so far. I've added the file and will ask for further information how to use it.
• TER_SCAO_dichroic.dat: The dichroic is used in transmission for NIR wavelengths (i.e. for science data). This is different from the current irdb definition. The action of the dichroic in LIST_MICADO_mirrors_static.dat has therefore been changed to transmission.
• filter curves: The new filter curves have been added as they were provided. The new narrow-band filter curves require a broad-band filter to be set in another filter wheel, otherwise there will be huge continuum leakage from broad wavelength ranges away from the actual filter location. This currently requires the user to be very careful with their setup (which should follow the actual MICADO setup closely). A proposal to make this more user friendly is deferred to Possible changes/improvements in MICADO setup #112.

[hugobuddel]

Thanks @oczoske.

About the curves: most of the original curves are such that they drop to 0.0 at the edges, and the new ones don't, some of the broad ones even go back up to 0.1 at the edges. I don't think this is technically problematic, so if you think this is fine, then I'm okay with it.

I'll let you decide whether you want to change the edges of the curves or not. Feel free to merge.

A tangential thought about tests: it would be good if we somehow start adding some tests that validate (and demonstrate to users) that ScopeSim actually performs according to Ric's specification. (Or not, in case of the antireflection coating.)

For the filters perhaps something like simulating an LSS observation of a featureless star, and then verify that the center and FWHM of the spectrum matches that of the filter. But that is beyond the scope of this PR.

The best would be if Ric (or Joost, etc.) accompanies these new tables with a (quantitative) way to demonstrate that everything works as anticipated, since they know best. We could then translate that to Python tests.

[oczoske]

• about the curves not dropping to zero: That actually was a problem in the past when extrapolation was done by a constant function at the level of the last point. I think this was subsequently replaced by extrapolation using zero, but I'd like to verify that in the code. As the data were provided by Ric I would like to touch the data themselves as little as possible. Update: There is no extrapolation, so I'll look at the data again.
• There seem to be some performance tests. At some point a test on limiting magnitudes failed (due to a clerical mistake - that missing underscore). I was actually a bit surprised that this test passes with new transmission data...

[hugobuddel]

• I agree that it would be good to keep our files as close to Ric's versions as we can. So perhaps it would indeed be better to extrapolate by using 0. I don't know what the state about this is.
• A while ago I relaxed the criteria for some of those tests, or marked them as xfail or skip, because I wanted no red tests. Maybe we can tighten them again. But if they would/should indeed fail with these new curves, then we should mark the tests as such (xfail).

[oczoske]

• Looking at the filter curves again, the only suspicious one is K-long, and even that is to be used in conjunction with K in the other filter wheel. Therefore, in the current setup everything is effectively forced to zero if used as described in the operational concept.
• Are those the tests in test_micado_background_flux.py ....XxxXXx or are there more? Removing the xfail marker there results in test_micado_background_flux.py .....FF..F with
  assert 39.870894060417434 == 30 ± 9.0e+00
assert 50.382763138765704 == 79 ± 2.4e+01
assert 6.804507054159926 == 11.109375 ± 3.3e+00
  These are not catastrophic failures, and as such probably rather tricky to figure out. The first thing that is needed is updated expectations, I guess.

[hugobuddel]

Those background tests are probably the main ones. I didn't really know what the correct numbers should be.

There is also this limiting magnitude test

irdb/MICADO/test_micado/test_micado_imaging.py

Lines 53 to 58 in bed2a8c

	# TODO: Fix J, or somehowe pytest xfail the test based on parametrization.
	@pytest.mark.parametrize(" fw1, fw2, r0, rics_lim_mag, abslim",
	[("J", "open", 1, 27.9, 1.0),
	("open", "H", 2, 27.5, 0.3),
	("open", "Ks", 2, 27.1, 0.3)])
	def test_MCAO_IMG_4mas(self, fw1, fw2, r0, rics_lim_mag, abslim, ao_mode="MCAO"):

where I changed the limit for the J-band.

Shall we merge this?

Next Tuesday me and @teutoburg are going to make some new releases of the ScopeSim-related software packages, and then maybe also of the IRDB packages. Do we want this in before that?

[oczoske]

I've just gone ahead and merged it. Good to have new releases out - this includes all of the MICADO requests for spectroscopy for now.