Add iue miri lrs and improve iue support

docs/code_capabilities.rst

@@ -83,9 +83,12 @@ The member functions of SpecData include:
 * read_spectra: generic read for a spectrum in a specifically formatted FITS file
 * read_fuse: read a FUSE spectrum
 * read_iue: read an IUE spectrum (includes cutting data > 3200 A)
-* read_stis: read a Hubble/STIS spectrum
+* read_stis: read a Hubble/STIS spectrum (either UV or optical range)
 * read_spex: read a IRTF/SpeX spectrum (includes scaling by corfacs)
 * read_irs: read a Spitzer/IRS spectrum (includes scaling by corfacs, cutting above some wavelength)
+* read_nircam_ss: read in NIRCam/slitless spectrum
+* read_miri_lrs: read in Webb/MIRI LRS spectrum
+* read_miri_ifu: read in Webb/MIRI MRS spectrum
 * rebin_constres: rebin spectrum to a constant input resolution
 
 ExtData

docs/data_formats.rst

@@ -4,19 +4,90 @@
 Data Formats
 ############
 
+The data for each stars is stored in a combination of an ASCII "DAT" file
+and FITS "spectra" files.
+
 DAT file
 ========
 
-Document the format of the DAT file
+The ASCII file is the only required file.  This file gives the photometry
+and links to the FITS files that contain the spectroscopy.  Many dat files
+for stars used in literature dust extinction curves are provided in the 
+`extstar_data repository <https://github.com/karllark/extstar_data>_`.
+
+An example of
+such a file (for a star from Valencic et al. 2004) that gives the Johnson photometry, 
+location of IUE spectrum, and spectral types is below.  This includes the
+photometry and associated uncertainties in Vega magnitudes.  Comments taking
+an entire line are allowed if they start with a "#" character.
+
+::
+
+    # data file for observations of HD 38023
+    V = 8.870 +/- 0.020
+    B = 9.190 +/- 0.030
+    U = 8.870 +/- 0.030
+    J = 8.080 +/- 0.020
+    H = 7.990 +/- 0.060
+    K = 7.900 +/- 0.020
+    IUE = IUE_Data/hd038023_iue.fits
+    sptype = B4V
+    uvsptype = B4IV
+
+A more complicated example giving photometry and spectroscopy from different sources
+is below (from a star from Gordon et al. 2021; Decleir et al. 2022 among other studies).
+The Johnson photometry is provide mainly as colors in this example.  Photometry
+rom IRAC/IRS/MIPS is provide in mJy.  The different photometry units are converted to the
+measure_extinction internal units when read into StarData.  In addition, there are comments
+provide after data using the ";" character.
+
+::
+
+    # data file for observations of HD 29647
+    V = 8.31 +/- 0.017 ; UBVRI ref = Slutskij, Stalbovskij & Shevchenko 1980 (1980SvAL....6..397S)
+    (U-B) = 0.47 +/- 0.030
+    (B-V) = 0.91 +/- 0.019
+    (V-R) = 0.96 +/- 0.024
+    (V-I) = 1.71 +/- 0.030
+    J = 5.960 +/- 0.030 ; JHK ref = IRSA 2MASS All-Sky Point Source Catalog
+    H = 5.593 +/- 0.021
+    K = 5.363 +/- 0.020
+    # IRAC 1 and 2 seem off
+    IRAC1 = 2.23E+03 +/- 4.45E+01 mJy
+    IRAC2 = 1.50E+03 +/- 3.00E+01 mJy
+    IRAC3 = 1.06E+03 +/- 2.16E+01 mJy
+    IRAC4 = 5.42E+02 +/- 1.09E+01 mJy
+    IRS15 = 3.14E+02 +/- 6.51E+00 mJy
+    MIPS24 = 6.76E+01 +/- 1.89E+00 mJy
+    # WISE have extended source contamination, contaminated by scattered moonlight
+    # WISE 1, 2 and 3 spurious detection of scattered light halo
+    # WISE 1 and 2 seem off, have saturated pixels
+    WISE1 = 5.179 +/- 0.150 ; WISE ref = IRSA AllWISE Source Catalog
+    WISE2 = 4.905 +/- 0.068
+    WISE3 = 4.882 +/- 0.023
+    WISE4 = 3.821 +/- 0.033
+    IUE = IUE_Data/hd029647_iue.fits
+    SpeX_SXD = SpeX_Data/hd029647_SXD_spex.fits
+    SpeX_LXD = SpeX_Data/hd029647_LXD_spex.fits
+    IRS = IRS_Data/hd029647_irs.fits
+    sptype = B7IV ; sptype ref = Murakawa, Tamura & Nagata 2000 (2000ApJS..128..603M)
+    uvsptype = B8III ; uvsptype ref = Valencic+2004 (2004ApJ...616..912V)
+
 
 Spectra
 =======
 
-FITS binary table files to give the spectra.
-Columns, header, etc.
+The spectra are stored in FITS files with standard columns and wavelength grids.
+The standard columns are "WAVELENGTH", "FLUX", "SIGMA", and "NPTS".  These files
+are created using the "merge_xxx_spec" functions provide in
+:class:`~measure_extinction.merge_obsspec`.  These ensure that all the spectra
+from a specific source (e.g., IUE) have the same wavelength grid, units, and standard
+columns.
 
-Ingest of spectra in to format needed for this package.
-Merge spectra and put on a common wavelength grid.
+For some of the types of spectra (e.g., IUE, STIS, SpeX, NIRCam SS), 
+there are commandline code in the `measure_extinction/utils` subdirectory to
+create such files from the commandline.
 
-IUE example?  Large + Small aperture observations give higher
-dynamic range.
+When using a stellar model for the comparison, each type of spectra supported is 
+simulated/mocked using code in the `measure_extinction/utils` subdirectory. 
+See :ref:`model standards`.

docs/index.rst

@@ -30,8 +30,8 @@ User Documentation
 
    Extinction Calculation <calc_extinction.rst>
    Capabilities <code_capabilities.rst>
+   Data Formats <data_formats.rst>
    Plotting <plotting.rst>
-   Observed Data Formats <data_formats.rst>
    Stellar Models as Standards <model_standards.rst>
 
 Reporting Issues

docs/model_standards.rst

@@ -1,3 +1,5 @@
+.. _model standards:
+
 ===========================
 Stellar Models as Standards
 ===========================

measure_extinction/merge_obsspec.py

@@ -8,6 +8,7 @@
     "merge_spex_obsspec",
     "merge_nircam_ss_obsspec",
     "merge_irs_obsspec",
+    "merge_miri_lrs_obsspec",
     "merge_miri_ifu_obsspec",
 ]
 
@@ -50,68 +51,6 @@ def _wavegrid(resolution, wave_range):
     return (full_wave, full_wave_min, full_wave_max)
 
 
-def merge_iue_obsspec(obstables, output_resolution=1000):
-    """
-    Merge one or more IUE 1D spectra into a single spectrum
-    on a uniform wavelength scale
-
-    Parameters
-    ----------
-    obstables : list of astropy Table objects
-        list of tables containing the observed IUE spectra
-        usually the result of reading tables
-
-    output_resolution : float
-        output resolution of spectra
-        input spectrum assumed to be at the appropriate resolution
-
-    Returns
-    -------
-    output_table : astropy Table object
-        merged spectra
-    """
-    wave_range = [1000.0, 3400.0] * u.angstrom
-
-    iwave_range = wave_range.to(u.angstrom).value
-    full_wave, full_wave_min, full_wave_max = _wavegrid(output_resolution, iwave_range)
-
-    n_waves = len(full_wave)
-    full_flux = np.zeros((n_waves), dtype=float)
-    full_unc = np.zeros((n_waves), dtype=float)
-    full_npts = np.zeros((n_waves), dtype=int)
-    for ctable in obstables:
-        # may want to add in the SYS-ERROR, but need to be careful
-        # to propagate it correctly, SYS-ERROR will not reduce with
-        # multiple spectra or measurements in a wavelength bin
-        cuncs = ctable["STAT-ERROR"].data
-        cwaves = ctable["WAVELENGTH"].data
-        cfluxes = ctable["FLUX"].data
-        cnpts = ctable["NPTS"].data
-        for k in range(n_waves):
-            (indxs,) = np.where(
-                (cwaves >= full_wave_min[k]) & (cwaves < full_wave_max[k]) & (cnpts > 0)
-            )
-            if len(indxs) > 0:
-                weights = 1.0 / np.square(cuncs[indxs])
-                full_flux[k] += np.sum(weights * cfluxes[indxs])
-                full_unc[k] += np.sum(weights)
-                full_npts[k] += len(indxs)
-
-    # divide by the net weights
-    (indxs,) = np.where(full_npts > 0)
-    if len(indxs) > 0:
-        full_flux[indxs] /= full_unc[indxs]
-        full_unc[indxs] = np.sqrt(1.0 / full_unc[indxs])
-
-    otable = Table()
-    otable["WAVELENGTH"] = Column(full_wave, unit=u.angstrom)
-    otable["FLUX"] = Column(full_flux, unit=u.erg / (u.s * u.cm * u.cm * u.angstrom))
-    otable["SIGMA"] = Column(full_unc, unit=u.erg / (u.s * u.cm * u.cm * u.angstrom))
-    otable["NPTS"] = Column(full_npts)
-
-    return otable
-
-
 def merge_stis_obsspec(obstables, waveregion="UV", output_resolution=1000):
     """
     Merge one or more STIS 1D spectra into a single spectrum
@@ -345,6 +284,34 @@ def merge_gen_obsspec(obstables, wave_range, output_resolution=100):
     return otable
 
 
+def merge_iue_obsspec(obstables, output_resolution=1000):
+    """
+    Merge one or more IUE 1D spectra into a single spectrum
+    on a uniform wavelength scale
+
+    Parameters
+    ----------
+    obstables : list of astropy Table objects
+        list of tables containing the observed IUE spectra
+        usually the result of reading tables
+
+    output_resolution : float
+        output resolution of spectra
+        input spectrum assumed to be at the appropriate resolution
+
+    Returns
+    -------
+    output_table : astropy Table object
+        merged spectra
+    """
+    wave_range = [1000.0, 3400.0] * u.angstrom
+
+    otable = merge_gen_obsspec(
+        obstables, wave_range, output_resolution=output_resolution,
+    )
+    return otable
+
+
 def merge_irs_obsspec(obstables, output_resolution=150):
     """
     Merge one or more Spitzer IRS 1D spectra into a single spectrum
@@ -426,6 +393,33 @@ def merge_nircam_ss_obsspec(obstables, output_resolution=1600):
     return otable
 
 
+def merge_miri_lrs_obsspec(obstables, output_resolution=100):
+    """
+    Merge one or more MIRI LRS spectra into a single spectrum
+    on a uniform wavelength scale
+
+    Parameters
+    ----------
+    obstables : list of astropy Table objects
+        list of tables containing the observed IRS spectra
+        usually the result of reading tables
+
+    output_resolution : float
+        output resolution of spectra
+        input spectrum assumed to be at the appropriate resolution
+
+    Returns
+    -------
+    output_table : astropy Table object
+        merged spectra
+    """
+    wave_range = [5.0, 13.0] * u.micron
+    otable = merge_gen_obsspec(
+        obstables, wave_range, output_resolution=output_resolution,
+    )
+    return otable
+
+
 def merge_miri_ifu_obsspec(obstables, output_resolution=3000):
     """
     Merge one or more MIRI IFU 1D spectra into a single spectrum

measure_extinction/plotting/plot_ext.py

@@ -1,8 +1,3 @@
-#!/usr/bin/env python
-
-from __future__ import absolute_import, division, print_function, unicode_literals
-
-import pkg_resources
 import argparse
 import warnings
 import matplotlib.pyplot as plt
@@ -11,6 +6,7 @@
 import pandas as pd
 import os
 
+from measure_extinction.utils.helpers import get_datapath
 from measure_extinction.extdata import ExtData
 from dust_extinction.parameter_averages import CCM89
 
@@ -92,8 +88,9 @@ def plot_average(
     Plots the average extinction curve
     """
     # read in the average extinction curve (if it exists)
-    if os.path.isfile(path + filename):
-        average = ExtData(path + filename)
+    fname = f"{path}/{filename}"
+    if os.path.isfile(fname):
+        average = ExtData(fname)
     else:
         warnings.warn(
             "An average extinction curve with the name "
@@ -331,8 +328,8 @@ def plot_HI(ax, wavenum=False):
     Indicates HI-lines on the plot
     """
     # read in HI-lines
-    path = pkg_resources.resource_filename("measure_extinction", "data/")
-    table = pd.read_table(path + "HI_lines.list", sep=r"\s+", comment="#")
+    data_path = get_datapath()
+    table = pd.read_table(f"{data_path}/HI_lines.list", sep=r"\s+", comment="#")
     # group lines by series
     series_groups = table.groupby("n'")
     colors = plt.get_cmap("tab10")
@@ -805,7 +802,7 @@ def main():
     parser.add_argument(
         "--path",
         help="path to the data files",
-        default=pkg_resources.resource_filename("measure_extinction", "data/"),
+        default=get_datapath(),
     )
     parser.add_argument("--alax", help="plot A(lambda)/A(X)", action="store_true")
     parser.add_argument(

measure_extinction/plotting/plot_spec.py

@@ -1,14 +1,10 @@
-#!/usr/bin/env python
-
-from __future__ import absolute_import, division, print_function, unicode_literals
-
-import pkg_resources
 import argparse
 import matplotlib.pyplot as plt
 import numpy as np
 import astropy.units as u
 import pandas as pd
 
+from measure_extinction.utils.helpers import get_datapath
 from measure_extinction.stardata import StarData
 
 
@@ -26,7 +22,7 @@ def plot_HI(path, ax):
     Indicates HI-lines on the plot
     """
     # read in HI-lines
-    table = pd.read_table(path + "HI_lines.list", sep=r"\s+", comment="#")
+    table = pd.read_table(f"{path}/HI_lines.list", sep=r"\s+", comment="#")
     # group lines by series
     series_groups = table.groupby("n'")
     colors = plt.get_cmap("tab10")
@@ -479,7 +475,7 @@ def main():
     parser.add_argument(
         "--path",
         help="path to the data files",
-        default=pkg_resources.resource_filename("measure_extinction", "data/"),
+        default=get_datapath(),
     )
     parser.add_argument("--mlam4", help="plot lambda^4*F(lambda)", action="store_true")
     parser.add_argument("--HI_lines", help="indicate the HI-lines", action="store_true")