Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let's get all the changes we've done so far on the main branch. Are we ready? #35

Closed
wants to merge 39 commits into from

Commits on Sep 30, 2023

  1. Configuration menu
    Copy the full SHA
    7210ee8 View commit details
    Browse the repository at this point in the history
  2. Remove DATA_VERSION conditionals

    DATA_VERSION is a constant set to 3, so only one conditional branch is
    used. To use older versions of the data, use older versions of the code.
    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    d6425a6 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    997b547 View commit details
    Browse the repository at this point in the history
  4. Replace relative paths with absolute paths

    You no longer have to be in a particular directory to run
    `update_data.py`.
    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    27200b6 View commit details
    Browse the repository at this point in the history
  5. Add dateutil, pillow deps

    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    62a68ba View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    c169cc9 View commit details
    Browse the repository at this point in the history
  7. Stub out mypy config

    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    7fa1dac View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    0c65836 View commit details
    Browse the repository at this point in the history
  9. Narrow scope of some imports

    I feel this makes the code more readable. We know what we're using from
    each module by looking at the import. And module names are not in our
    code, meaning less mental overhead is required to parse the code.
    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    db8a21e View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    163d5c7 View commit details
    Browse the repository at this point in the history
  11. Write database to optional external location

    I'm aiming for max compatibility with the old code. What made the most
    sense to me was having a new `database` directory which can be either
    internal or external. The `data` directory already contains internal
    stuff, so I felt they should be separated.
    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    706a78f View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    fcda0ad View commit details
    Browse the repository at this point in the history
  13. Lock environment

    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    51fa2be View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    647ff44 View commit details
    Browse the repository at this point in the history
  15. WIP operations doc

    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    65b773f View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2023

  1. Configuration menu
    Copy the full SHA
    baee844 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    014d9f7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    037b5f3 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    64c573b View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. Configuration menu
    Copy the full SHA
    bb7af78 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. Configuration menu
    Copy the full SHA
    3051567 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2023

  1. Configuration menu
    Copy the full SHA
    3d79679 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3fb8a38 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4744eaf View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. Fixing deprecated usages of old numpy dtypes.

    Removed references to:
    - "numpy.int" and "numpy.long" (replaced by "int"),
    - "numpy.float" (replaced by "float")
    - "numpy.bool" (replaced by "bool")
    
    References to specific type sizes (e.g. "numpy.float32" or "numpy.int16") are not deprecated, and remain in place.
    Removed the requirement in environmental.yaml to suggest older versions of numpy, for which this deprecated code didn't break things. This code should now be compatible with the latest versions of numpy (as of this commit).
    mmacferrin authored and mfisher87 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    50f9fb9 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. Minor bug fixes and improved comments.

    * generate_antarctica_today_map.py: Fixed small bug (line 1727) that returned the wrong year-number if you generated a map on the very last day of the melt season (April 30).
    * map_filedata.py: Fixed pointers to basin outline shapefiles, which are in the "basins" sub-directory of the "qgis" folder.
    * melt_array_picklefile.py: Fixed an incorrect raw-string typing, and expanded the docstring of the _filter_out_erroneous_swaths() function to better explain what that function is doing.
    * nsidc_download_Tb_data.py: Removed extraneous import statements. Also coded around the time-zone issue in Earthdata to avoid runtime errors, and added code to avoid attempting to download data if no available files were identified.
    * write_NSIDC_bin_to_gtif.py: Added a call to osr.UseExceptions() to improve forward-compatibility, and avoid constantly seeing a GDAL run-time Warning about that.
    mmacferrin committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    ea0d2ac View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2023

  1. Updated "update_data.py" to use work with nsidc-0080 v2 data (from v1…

    …). Also created new script "generate_plots_for_given_day.py" with a simple CLI to generate plots for any past day in the time-series.
    mmacferrin committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    2076a10 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    fb65820 View commit details
    Browse the repository at this point in the history
  3. Added an optional parameter to limit & filter the download of Tb brig…

    …htness-temperature data to only within the span of the melt season (Oct 1st thru April 30th) in any given year. This avoids unnecessary downloads of Tb data for which melt is not calculated. (Winter Tb data is used to calibrate the microwave model with thresholds for the following season, but melt is not calculated during the winter months.)
    mmacferrin committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    2fe965d View commit details
    Browse the repository at this point in the history
  4. Ignore .nc and .bin files in the Tb/nsidc-0080 folder. These are to b…

    …e downloaded from NSIDC and stored locally only.
    mmacferrin committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    c79cf46 View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2023

  1. Configuration menu
    Copy the full SHA
    1f9c68c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7a39152 View commit details
    Browse the repository at this point in the history
  3. Fixed runtime bug in previous update to "update_data.py" that was cau…

    …sing it to miss needed updates after downloading most-recent Tb data from NSIDC.
    
    Also, made the default behavior when "gathering" files into a single daily directory, to use symlinks rather than copyfile() in order to reduce redundant disk usage of plot images.
    mmacferrin committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    659587f View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2023

  1. Configuration menu
    Copy the full SHA
    6b911ed View commit details
    Browse the repository at this point in the history
  2. Quick bug fix (mis-named variable).

    Also a call to matplotlib.pyplot.clear('all') to close hanging matplotlib plots between each region iteration.
    mmacferrin committed Dec 17, 2023
    Configuration menu
    Copy the full SHA
    aa3edaa View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2023

  1. Flipped a parameter to *not* make it use symlinks when copying everyt…

    …hing into the "daily_gathered_plots" directory.
    
    It was meant to save space but caused problems on some platforms when trying to upload the images online.
    mmacferrin committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    13a7ed8 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. Configuration menu
    Copy the full SHA
    1e21d83 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f069e9a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2b417c7 View commit details
    Browse the repository at this point in the history