Implement 1D wavelength calibration classes #162

rosteen · 2023-01-26T22:39:56Z

This implements the initial work needed for 1D wavelength calibration, defaulting to a basic linear fit to the input wavelength/pixel pairs. Does not implement autoidentification of lines or input of a preset named line list. I have a little bit of refinement left to do (e.g., making sure the internals are consistent on whether pixel values are stored as Quantities with u.pix) and need to add tests, but I figured it's to a point where I should get other eyes on it to make sure I'm fulfilling the consensus reached in the discussion on #152.

The following code snippet is what I was using to test this:

import astropy.units as u
import numpy as np
from specutils import Spectrum1D
from wavelength_calibration import WavelengthCalibration1D

flux = np.random.random(50)*u.Jy
sa = np.arange(0,50)*u.pix
spec = Spectrum1D(flux, spectral_axis = sa)
test = WavelengthCalibration1D(spec, [(0,5000*u.AA),(10,5100*u.AA),(20,5198*u.AA),(30,5305*u.AA)])
spec2 = test.apply_to_spectrum(spec)
spec2

codecov · 2023-01-26T22:43:15Z

Codecov Report

Merging #162 (d573219) into main (7043f78) will increase coverage by 1.78%.
The diff coverage is 90.32%.

❗ Current head d573219 differs from pull request most recent head 82170a5. Consider uploading reports for the commit 82170a5 to get more accurate results

@@            Coverage Diff             @@
##             main     #162      +/-   ##
==========================================
+ Coverage   76.58%   78.36%   +1.78%     
==========================================
  Files           9       10       +1     
  Lines         709      809     +100     
==========================================
+ Hits          543      634      +91     
- Misses        166      175       +9

Impacted Files	Coverage Δ
specreduce/wavelength_calibration.py	`90.21% <90.21%> (ø)`
specreduce/__init__.py	`100.00% <100.00%> (ø)`

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

…hCalibration1D, add tests

kecnry · 2023-01-31T18:23:17Z

specreduce/wavelength_calibration.py

+    @classmethod
+    def by_name(cls, input_spectrum, line_name, pixel,
+                refinement_method=None, refinement_kwargs={}):
+        # this will do wavelength lookup and passes that wavelength to __init__ allowing the user
+        # to call CalibrationLine.by_name(spectrum, 'H_alpha', pixel=40)
+        # We could also have a type-check for ``wavelength`` above, this just feels more verbose
+        return cls(...)


should we just remove this for now?

kecnry · 2023-01-31T18:24:58Z

specreduce/wavelength_calibration.py

+        if self.refinement_method in ("gaussian", "min", "max"):
+            if 'range' not in self.refinement_kwargs:
+                raise ValueError(f"You must define 'range' in refinement_kwargs to use "
+                                 f"{self.refinement_method} refinement.")
+        elif self.refinement_method == "gradient" and 'direction' not in self.refinement_kwargs:
+            raise ValueError("You must define 'direction' in refinement_kwargs to use "
+                             "gradient refinement")


do we want defaults for these options or is it too hard to make reasonable defaults that can be generalized?

I would be lean toward making gaussian the default, personally.

The issue is that the range parameter depends heavily on the properties of your spectrum/array, I think. Hence why I left the default as no refinement.

Sorry, I meant automatic defaults for refinement_kwargs, definitely refinement_method-dependent, and perhaps even estimated from the input_spectrum (not sure how complicated that would get - definitely could be a follow-up effort... I just feel like users are constantly going to be seeing this error message).

kecnry · 2023-01-31T18:27:22Z

specreduce/wavelength_calibration.py

+
+        return refined_pixel
+
+    @cached_property


should probably be coupled with custom setters for the input spectrum and options to clear the cache

specreduce/wavelength_calibration.py

kecnry · 2023-01-31T18:31:46Z

specreduce/wavelength_calibration.py

+        # useful for plotting over spectrum to change input lines/refinement options
+        return [(line.wavelength, line.refined_pixel) for line in self.lines]
+
+    @cached_property


cache needs to be cleared when appropriate

That can be accomplished by deleting the cached properties from the class __dict__, right? I haven't use cached_property before

exactly. In jdaviz I wrote a helper method that might also be useful in this case

Alright, I implemented what I think you had in mind, take a look.

that looks good to me, but I think there are other cases that need to clear too (wcs cache needs to clear for a change in pretty much any of the original inputs, I think). I wonder if there's a way to generalize away this logic so we don't need to have all this boilerplate code 🤔 (definitely can generalize later - it would probably take a custom __setattr__ which might be a little hacky-feeling)

Also now clears the WCS cache on changing the lines or model attributes. Do you think that suffices for now? I imagine that at some point people would just make a new WavelengthCalibration1D rather than updating an existing one.

kecnry · 2023-01-31T18:33:29Z

specreduce/wavelength_calibration.py

+
+        return wcsobj
+
+    def apply_to_spectrum(self, spectrum=None):


Do we want __call__ instead, to just call this, to do something else, or to not exist? I think we will eventually want calling to do something since that pattern is supported through most/all of the other specreduce operations.

I remembered @eteq was somewhat against having a __call__ at all, so I had left it out for now. If it's more consistent with the way the rest of the package works, then I guess we can think about what makes the most sense. Having it call this method on the default (input) spectrum seems like an ok behavior to me, but I don't feel strongly about it.

Ok, I think we'll want __call__ to do something eventually, but I'm fine deferring that until the rest of the API begins to converge in follow-up efforts.

a workflow we will need to support is where we iterate the fitting process. one would start with a small number of matched lines and a simple model, use the model to match more lines, re-fit, tweak model, match more lines, re-fit/tweak/match, and repeat until a convergence criteria is met. my view is that this class/function should be within that loop, but not contain that loop. i don't see where a __call__ fits into that scheme and i think a apply_to_spectrum() method is a clearer way of applying the results to a spectrum.

…implemented method

kecnry · 2023-02-03T20:16:24Z

specreduce/wavelength_calibration.py

+        y = [line[0].value for line in self.refined_pixels] * self.spectral_unit
+
+        # Fit the model
+        self._model = self.fitter(self._model, x, y)


does this prevent successive fitting calls?

I don't think so...I guess successive calls would be starting from the previous fitted parameter values, but I don't think it prevents fitting. Do you think having a separate _fitted_model parameter would be better?

I'm not sure 🤷, I think the way you have it - assuming the output can be passed as the input - is both quite handy but also maybe unexpected behavior unless we document carefully.

kecnry · 2023-02-03T20:17:18Z

specreduce/wavelength_calibration.py

+        if fitter is None:
+            if self.model.linear:
+                self.fitter = LinearLSQFitter(calc_uncertainties=True)
+            else:
+                self.fitter = LMLSQFitter(calc_uncertainties=True)


what if instead self._fitter was stored as None and then a fitter property defaults based on the current value of self.model (so that if model is changed after initialization, the default can change to match)?

I like this idea, I'll implement momentarily.

rosteen · 2023-02-07T17:13:47Z

@kecnry I added docs - please look them over. I'm going to open an issue to revisit whether we want to store the fitted model in a separate property from the initial model.

tepickering

a few high-level comments:

there definitely needs to be a concept of a LineList. for example, the HeNeAr lamps that are very commonly used for ground-based optical data provide several hundred possible calibration lines. different calibration lamps are often combined so there needs to be a way to manage and combine lists based on configuration. handling such sets of calibration lines is a far more common use case than managing individual ones.
i'm not sure why the CalibrationLine class needs pixel position refinement. the locating and centroiding of potential calibration lines in pixel-space should happen in a separate, previous step.
iterative fitting is something we'll likely want to implement. basically, start with min_order and increase order until either the fit RMS stops improving or max_order is reached. it's probably something that can be added later, but we should be cognizant of the future need.

rosteen · 2023-02-21T16:49:31Z

Thanks for the thoughts, Tim. Your points 1 and 3 are planned for future development, we're just trying to be iterative about this and not have one massive PR that takes 6 months to finish. Your second comment is the only one that might be an actual sticking point, I think, but I don't think that the current class precludes the workflow it sounds like you're imagining. If the pixel positions of the lines have already been refined, you can simply leave the refinement args in the CalibrationLine class to their defaults, which don't do any refinement. As you mentioned with the LineList point, the plan is to add more automated options later, but this initial work enables the workflow of manually identifying some lines in the cal spectrum and using that starting point to do the fit (with or without refinement). I see the workflow you seem to want as compatible enhancements to this initial work, not as fundamentally incompatible. Do you disagree?

tepickering · 2023-02-23T21:22:59Z

my main concern, especially with point 2, is that sub-optimal architecture that gets baked in early causes significant issues down the road. wavelength calibration fundamentally requires managing lists of lines because you need multiple lines to formulate a wavelength solution. as implemented here, the CalibrationLine class is mixing pixel and wavelength space for each individual line. it might make sense for the simple, manual examples here, but i'm not seeing how it will be at all beneficial when dealing with large lists of lines and no prior knowledge of pixel->wavelength mapping.

in my head and how most other implementations work, you first create an extracted line list from an extracted spectrum. the peak-finding and accurate centroiding of the lines is done in the creation of that. this can fundamentally just be a list of pixel positions, but could optionally include line fluxes as well. the fluxes could be used to set thresholds for what lines to use or can be used by matching algorithms.

then you have a list of wavelengths of potential calibration lines. usually these are archived catalogs, but it could be a manual list. again, fundamentally just a list of wavelengths, but could include other optional info like relative strengths or whether the wavelengths are vacuum or air.

i should note that pypeit and DRAGONS use astropy.table internally for handling these kinds of lists. we may want to do likewise because i can definitely foresee potential needs for functionality like vstack.

where a CalibrationLine class might come into play is where lines are matched between the pixel list and wavelength list. however, something list-like would make more sense since you're passing a list of wavelength/pixel pairs to the fitting routine. iterating through a bunch of CalibrationLine instances to get that information would be a lot slower than just grabbing a couple of lists or numpy arrays from a Table or similar.

this matching is a process and there are many ways to do it. this initial example of manual matching is fine and there will always be a use case for that. my contribution will be porting over automated and template-based matching. the output from any of the methods should be the same format of wavelength/pixel pairs. so there should be some kind of class hierarchy of LineMatcher or something like that to cover these. maybe not now, but definitely once i add my stuff.

i'll see if i can put these words into a clearer flowchart...

tepickering · 2023-04-06T22:19:18Z

awesome! i'll take a quick look tonight and a deeper dive tomorrow...

tepickering · 2023-04-06T23:28:55Z

the readthedocs build error should get fixed once this is rebased to include #165 after it's merged. i am deprecating specreduce_data as a python package and making into simply a data repository.

kecnry

Just a few comments/questions from a quick skim of the code. Are there any examples for different workflows that can be used for testing or should we just refer to the tests for now?

kecnry · 2023-04-07T13:31:38Z

specreduce/wavelength_calibration.py

+        catalog: list, str, optional
+            The name of a catalog of line wavelengths to load and use in automated and
+            template-matching line matching.


could this be moved to a class method? So WavelengthCalibration1D.from_line_list(line_list) calls WavelengthCalibration1D(...)?

what i implemented in #165 returns a line catalog as a QTable. so this should be able to accept that and make sure there's a wave or wavelength column.

specreduce/wavelength_calibration.py

kecnry · 2023-04-07T13:34:55Z

specreduce/wavelength_calibration.py

+                line_wavelengths.value.sort()
+                self._line_list["wavelength"] = line_wavelengths


does line_wavelengths support frequencies? Would the sorting need to be flipped in this case? What if they're mixed units?

I'm not sure how common it is to use frequencies for this (my assumption was "never" but I guess someone might want to), but we could support it. I can also add a check to make sure they're all in the same unit.

Actually, since this requires line_wavelengths to be a Quantity or QTable (rather than allowing a list of Quantity), it's already guaranteed that all elements have the same unit.

@kecnry I implemented reverse-sorting if frequencies are provided, although that does make the variable name line_wavelengths a bit of a misnomer.

my first take is that since this package is geared towards 2D optical/IR data obtained with CCD-like devices, i would stick with wavelengths internally and not worry about frequencies at all. at least for simplicity at this stage of development. if users have dispersed data with units of frequency, there are existing utilities to transform it to wavelengths. i can imagine possible use cases for 2D spatial vs frequency data in radio astronomy, but i've never seen it. radio data is either 1D (single-dish) or 3D (scanned single-dish or interferometry). eV or wavenumber might be more likely candidates for non-wavelength inputs, but they can also easily be transformed to wavelength.

specreduce/wavelength_calibration.py

kecnry · 2023-04-07T13:39:01Z

specreduce/conftest.py

+@pytest.fixture
+def spec1d():
+    np.random.seed(7)
+    flux = np.random.random(50)*u.Jy
+    sa = np.arange(0, 50)*u.pix
+    spec = Spectrum1D(flux, spectral_axis=sa)
+    return spec
+
+
+@pytest.fixture
+def spec1d_with_emission_line():
+    np.random.seed(7)
+    sa = np.arange(0, 200)*u.pix
+    flux = (np.random.randn(200) +
+            10*np.exp(-0.01*((sa.value-130)**2)) +
+            sa.value/100) * u.Jy
+    spec = Spectrum1D(flux, spectral_axis=sa)
+    return spec
+
+
+@pytest.fixture
+def spec1d_with_absorption_line():
+    np.random.seed(7)
+    sa = np.arange(0, 200)*u.pix
+    flux = (np.random.randn(200) -
+            10*np.exp(-0.01*((sa.value-130)**2)) +
+            sa.value/100) * u.Jy
+    spec = Spectrum1D(flux, spectral_axis=sa)
+    return spec
+
+


would this benefit from using the synthetic spectra from #165?

Yes, but I think using that to expand/improve the tests can be another PR once both of these are merged.

Co-authored-by: Kyle Conroy <[email protected]>

rosteen · 2023-04-07T20:57:48Z

@tepickering Think you'll get a chance to take a deeper look by Monday afternoon or so? I'd like to close this out in line with our Sprint end.

tepickering · 2023-04-07T21:09:22Z

i will take a deeper dive later today, if not over the weekend. it was a little hard to follow the changes on github due to missing test coverage so i'll pull in a local copy.

tepickering · 2023-04-08T01:08:18Z

docs/wavelength_calibration.rst

+
+The example above uses the default model (`~astropy.modeling.functional_models.Linear1D`)
+to fit the input spectral lines, and then applies the calculated WCS solution to a second
+spectrum (``science_spectrum``). Any other ``astropy`` model can be provided as the


maybe specify "any 1D model"

tepickering · 2023-04-08T01:13:09Z

docs/wavelength_calibration.rst

+    from astropy.table import QTable
+    pixels = [10, 20, 30, 40]*u.pix
+    wavelength = [5340, 5410, 5476, 5543]*u.AA
+    line_list = QTable([pixels, wavelength], names=["pixel_center", "wavelength"])


nitpick: i'd call this matched_line_list so as not to be confused with linelist or line_list which is usually understood to mean something else (i.e. list of wavelengths of lines due to specific ion/molecule or mixture of of them).

tepickering

this is a much-improved implementation that gives us something to build on. i left some minor comments about some wording that should get sorted out. there'll be some meshing needed between this and what i've done in #165. that can happen in a future PR, though. the most substantive thing i'd like to see addressed before merging to cover the remaining missing lines with tests. there's not too many...

tepickering · 2023-04-08T01:19:44Z

specreduce/wavelength_calibration.py

+__all__ = ['WavelengthCalibration1D']
+
+
+def get_available_catalogs():


thus far there is built-in access to some legacy line lists/line catalogs in the specreduce-data repository and more well-curated ones in pypeit. since either way would require spidering through remote directories to find what's there, i think this is probably better left to the documentation rather than a function like this. you need to know some details about which one to use because they can be very instrument-dependent.

actually, i forgot that i did make a global variable, PYPEIT_CALIBRATION_LINELISTS, that does list which ones are available that we'd want to use. so after #165 is merged, this could just return that.

tepickering · 2023-04-08T01:21:47Z

specreduce/wavelength_calibration.py

+    """
+    ToDo: Code logic to combine the lines from multiple catalogs if needed
+    """
+    pass


i already implemented this in #165 for pypeit line lists. if you give a list of supported ions/molecules, it generates a combined line list.

Great, we'll definitely need a PR after these are both merged to combine their powers 😃

tepickering · 2023-04-08T01:25:03Z

specreduce/wavelength_calibration.py

+        """
+        input_spectrum: `~specutils.Spectrum1D`
+            A one-dimensional Spectrum1D calibration spectrum from an arc lamp or similar.
+        line_list: list, array, `~astropy.table.QTable`


i would call this line_pixels to make it clearer.

My thought here was that the user can provide the whole table, including both pixels and wavelengths, in which case it doesn't make sense to call it 'line_pixels. Maybe I'll change this slightly to have separate line_list and line_pixels arguments, where the former can be the whole table, and the latter is intended for just pixel values to then put internally into the table.

tepickering · 2023-04-08T01:29:12Z

specreduce/wavelength_calibration.py

+        catalog: list, str, optional
+            The name of a catalog of line wavelengths to load and use in automated and
+            template-matching line matching.


what i implemented in #165 returns a line catalog as a QTable. so this should be able to accept that and make sure there's a wave or wavelength column.

rosteen · 2023-04-10T21:31:35Z

Coverage is up to 90%, the rest is really dependent on integrating with #165 so I think it can wait. I think all I have left to address @tepickering's comments is to split out matched_line_list and line_pixels into separate input arguments - I think the way forward is to make them both optional but require at least one. Won't take long to do, but I ran out of time today.

major refactor

tepickering · 2023-04-11T17:26:40Z

i'm happy with the latest round of changes. as previously mentioned, further work to mesh functionality and vocabulary with #165 is best left to a subsequent PR.

rosteen · 2023-04-11T17:33:00Z

Excellent! Tests passed again after my update splitting out those two input arguments, let's get this in 🎉

tepickering · 2023-04-11T17:36:23Z

lgtm! if @kecnry concurs, feel free to merge. then i'll do the same for #165.

rosteen · 2023-04-11T17:55:41Z

I don't have the power to merge here, @kecnry can do it.

kecnry

I have some questions/concerns about the big-picture plan for this as catalogs and the 2d cases get implemented, but so long as the plan is to iterate further in the future, won't object to merging this as-is. I'd love to see the actual pseudocode examples updated for various science workflows though - just to see where this is heading in advance. @tepickering, how do you imagine the catalog interface would work with this? There is no refinement implemented here anymore, so how does the 2d case work?

I know we haven't committed to backwards-compatibility yet, but would it make sense to merge this into a development branch instead of main until we're ready for it to be (more) public facing (@tepickering)?

kecnry · 2023-04-11T18:50:00Z

specreduce/wavelength_calibration.py

+        Note that either ``matched_line_list`` or ``line_pixels`` must be specified,
+        and if ``matched_line_list`` is not input, at least one of ``line_wavelengths``
+        or ``catalog`` must be specified.


this feels to me like it would be better as multiple classmethod, but we can revisit this as we iterate further on the API and various workflows as I recognize that might be a matter of taste:

WavelengthCalibration1D(input_spectrum, matched_line_list, model, fitter) (__init__)

WavelengthCalibration1D.from_lists(input_spectrum, line_pixels, line_wavelengths, model, fitter)

WavelengthCalibration1D.from_catalog(input_spectrum, catalog, model, fitter) (and perhaps line_pixels?)

This also would allow getting rid of a lot of code that just checks for validity of the combinations of all of these technically-optional but-sorta-required kwargs.

yeah, this is still kinda messy. i think we should actually keep WavelengthCalibration1D as simple as possible: just WavelengthCalibration1D(line_pixels, line_wavelengths, model, fitter) where line_pixels and line_wavelengths are list-like of the same length. there are previous steps required to find/centroid lines in a calibration spectrum (specutils provides tools for this) and then match some of those lines to known wavelengths to construct line_pixels and line_wavelengths (manually or via to-be-implemented automated techniques). i don't think we should wrap all of these steps into one big class. the interfacing with things like catalogs of known lines like i implement in #165 will happen at the line-matching stage.

anyway, it will be a lot easier to iterate on these ideas using realistic end-to-end examples once this work is merged with #165. the work may not be complete after these are merged, but nothing is broken so i don't think we need a separate develop branch`.

tepickering · 2023-04-11T22:38:25Z

docs/wavelength_calibration.rst

+	 from specreduce import WavelengthCalibration1D
+	 pixel_centers = [10, 22, 31, 43]
+	 wavelengths = [5340, 5410, 5476, 5543]*u.AA
+	 test_cal = WavelengthCalibration1D(lamp_spectrum, line_pixels=pixel_centers,


note that lamp_spectrum is not defined in this example.

tepickering

i've left several more substantive comments. i still think WavelengthCalibration1D is a bit too high-level, though a big step in the right direction. it works well enough to merge now and future work will be a lot easier with #165 merged in as well. being able to generate realistic data to prototype realistic workflows will be a significant benefit.

rosteen added 6 commits January 18, 2023 15:22

Working on line pixel refinement

d58f652

Testing/debugging CalibrationLine, starting on WavelengthCalibration

1f3483a

Working on model fitting and construction of resulting GWCS

c54aed7

Working through bugs/testing

5eb4aaa

Working initial implementation

baa706c

Codestyle

02ed67e

rosteen added 5 commits January 27, 2023 13:26

Match order of pixel/wavelength between CalibrationLine and Wavelengt…

1df8621

…hCalibration1D, add tests

Fix codestyle in test

eb5feea

Adding tests for CalibrationLine

6ea98e4

Fix codestlye

af4a523

Increase atol for gaussian center

568fd09

kecnry previously approved these changes Jan 31, 2023

View reviewed changes

rosteen added 5 commits February 1, 2023 11:21

Remove autoidentify for now, add Polynomial1D test

be35c09

Add cache clearing, avoid AstropyUserWarning

f3b1b9f

Add input_spectrum property and clear cache on setting it

8117864

Add default range if method is set but not range, remove check for un…

8f1535b

…implemented method

Also clear cache when updating lines or model

22e2d68

kecnry reviewed Feb 3, 2023

View reviewed changes

rosteen added 2 commits February 3, 2023 15:56

Move fitter default to wcs call

4a508e3

Keep storing fitter as None

a8ca23f

kecnry self-requested a review February 6, 2023 13:50

Add docs, improve importing of new classes

1941a26

rosteen mentioned this pull request Feb 7, 2023

[Discussion]: Store fitted model separately in WavelengthCalibration1D #164

Open

kecnry requested review from tepickering and eteq February 9, 2023 15:59

tepickering reviewed Feb 20, 2023

View reviewed changes

Codestyle

c3ba30d

kecnry reviewed Apr 7, 2023

View reviewed changes

rosteen and others added 5 commits April 7, 2023 11:29

Apply suggestions from code review

62c8025

Co-authored-by: Kyle Conroy <[email protected]>

Remove speactral_units arg, add docstring for fitter arg

12a8661

Update documentation, add NotImplementedError for catalog input

5063b5e

Reverse sort frequencies

b6a92f8

Codestyle

1aa62c6

tepickering reviewed Apr 8, 2023

View reviewed changes

tepickering requested changes Apr 8, 2023

View reviewed changes

rosteen added 4 commits April 10, 2023 15:52

Increase test coverage

7f31b61

Fix tests and bugs found by tests

45939a9

Codestyle

f92bac5

Add QTable to catalog options

d573219

tepickering approved these changes Apr 11, 2023

View reviewed changes

Finish addressing review comments

82170a5

kecnry reviewed Apr 11, 2023

View reviewed changes

tepickering reviewed Apr 11, 2023

View reviewed changes

tepickering approved these changes Apr 11, 2023

View reviewed changes

kecnry merged commit 345c452 into astropy:main Apr 12, 2023

		line_wavelengths.value.sort()
		self._line_list["wavelength"] = line_wavelengths

		__all__ = ['WavelengthCalibration1D']


		def get_available_catalogs():

Implement 1D wavelength calibration classes #162

Implement 1D wavelength calibration classes #162

Conversation

rosteen commented Jan 26, 2023 • edited Loading

codecov bot commented Jan 26, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rosteen commented Feb 7, 2023

tepickering left a comment

Choose a reason for hiding this comment

rosteen commented Feb 21, 2023 • edited Loading

tepickering commented Feb 23, 2023

tepickering commented Apr 6, 2023

tepickering commented Apr 6, 2023

kecnry left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rosteen commented Apr 7, 2023

tepickering commented Apr 7, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tepickering left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rosteen commented Apr 10, 2023 • edited Loading

tepickering commented Apr 11, 2023

rosteen commented Apr 11, 2023

tepickering commented Apr 11, 2023

rosteen commented Apr 11, 2023

kecnry left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tepickering left a comment

Choose a reason for hiding this comment

rosteen commented Jan 26, 2023 •

edited

Loading

codecov bot commented Jan 26, 2023 •

edited

Loading

rosteen commented Feb 21, 2023 •

edited

Loading

rosteen commented Apr 10, 2023 •

edited

Loading