Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 2 #40

Draft
wants to merge 81 commits into
base: main
Choose a base branch
from
Draft

Version 2 #40

wants to merge 81 commits into from

Commits on Sep 30, 2023

  1. Configuration menu
    Copy the full SHA
    7210ee8 View commit details
    Browse the repository at this point in the history
  2. Remove DATA_VERSION conditionals

    DATA_VERSION is a constant set to 3, so only one conditional branch is
    used. To use older versions of the data, use older versions of the code.
    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    d6425a6 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    997b547 View commit details
    Browse the repository at this point in the history
  4. Replace relative paths with absolute paths

    You no longer have to be in a particular directory to run
    `update_data.py`.
    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    27200b6 View commit details
    Browse the repository at this point in the history
  5. Add dateutil, pillow deps

    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    62a68ba View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    c169cc9 View commit details
    Browse the repository at this point in the history
  7. Stub out mypy config

    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    7fa1dac View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    0c65836 View commit details
    Browse the repository at this point in the history
  9. Narrow scope of some imports

    I feel this makes the code more readable. We know what we're using from
    each module by looking at the import. And module names are not in our
    code, meaning less mental overhead is required to parse the code.
    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    db8a21e View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    163d5c7 View commit details
    Browse the repository at this point in the history
  11. Write database to optional external location

    I'm aiming for max compatibility with the old code. What made the most
    sense to me was having a new `database` directory which can be either
    internal or external. The `data` directory already contains internal
    stuff, so I felt they should be separated.
    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    706a78f View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    fcda0ad View commit details
    Browse the repository at this point in the history
  13. Lock environment

    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    51fa2be View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    647ff44 View commit details
    Browse the repository at this point in the history
  15. WIP operations doc

    mfisher87 committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    65b773f View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2023

  1. Configuration menu
    Copy the full SHA
    baee844 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    014d9f7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    037b5f3 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    64c573b View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. Configuration menu
    Copy the full SHA
    bb7af78 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. Configuration menu
    Copy the full SHA
    3051567 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2023

  1. Configuration menu
    Copy the full SHA
    3d79679 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3fb8a38 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4744eaf View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. Fixing deprecated usages of old numpy dtypes.

    Removed references to:
    - "numpy.int" and "numpy.long" (replaced by "int"),
    - "numpy.float" (replaced by "float")
    - "numpy.bool" (replaced by "bool")
    
    References to specific type sizes (e.g. "numpy.float32" or "numpy.int16") are not deprecated, and remain in place.
    Removed the requirement in environmental.yaml to suggest older versions of numpy, for which this deprecated code didn't break things. This code should now be compatible with the latest versions of numpy (as of this commit).
    mmacferrin authored and mfisher87 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    50f9fb9 View commit details
    Browse the repository at this point in the history
  2. Fix linter complaints

    mfisher87 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    40bb97d View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. Minor bug fixes and improved comments.

    * generate_antarctica_today_map.py: Fixed small bug (line 1727) that returned the wrong year-number if you generated a map on the very last day of the melt season (April 30).
    * map_filedata.py: Fixed pointers to basin outline shapefiles, which are in the "basins" sub-directory of the "qgis" folder.
    * melt_array_picklefile.py: Fixed an incorrect raw-string typing, and expanded the docstring of the _filter_out_erroneous_swaths() function to better explain what that function is doing.
    * nsidc_download_Tb_data.py: Removed extraneous import statements. Also coded around the time-zone issue in Earthdata to avoid runtime errors, and added code to avoid attempting to download data if no available files were identified.
    * write_NSIDC_bin_to_gtif.py: Added a call to osr.UseExceptions() to improve forward-compatibility, and avoid constantly seeing a GDAL run-time Warning about that.
    mmacferrin committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    ea0d2ac View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2023

  1. Updated "update_data.py" to use work with nsidc-0080 v2 data (from v1…

    …). Also created new script "generate_plots_for_given_day.py" with a simple CLI to generate plots for any past day in the time-series.
    mmacferrin committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    2076a10 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    fb65820 View commit details
    Browse the repository at this point in the history
  3. Added an optional parameter to limit & filter the download of Tb brig…

    …htness-temperature data to only within the span of the melt season (Oct 1st thru April 30th) in any given year. This avoids unnecessary downloads of Tb data for which melt is not calculated. (Winter Tb data is used to calibrate the microwave model with thresholds for the following season, but melt is not calculated during the winter months.)
    mmacferrin committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    2fe965d View commit details
    Browse the repository at this point in the history
  4. Ignore .nc and .bin files in the Tb/nsidc-0080 folder. These are to b…

    …e downloaded from NSIDC and stored locally only.
    mmacferrin committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    c79cf46 View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2023

  1. Configuration menu
    Copy the full SHA
    1f9c68c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7a39152 View commit details
    Browse the repository at this point in the history
  3. Fixed runtime bug in previous update to "update_data.py" that was cau…

    …sing it to miss needed updates after downloading most-recent Tb data from NSIDC.
    
    Also, made the default behavior when "gathering" files into a single daily directory, to use symlinks rather than copyfile() in order to reduce redundant disk usage of plot images.
    mmacferrin committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    659587f View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2023

  1. Configuration menu
    Copy the full SHA
    6b911ed View commit details
    Browse the repository at this point in the history
  2. Quick bug fix (mis-named variable).

    Also a call to matplotlib.pyplot.clear('all') to close hanging matplotlib plots between each region iteration.
    mmacferrin committed Dec 17, 2023
    Configuration menu
    Copy the full SHA
    aa3edaa View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2023

  1. Flipped a parameter to *not* make it use symlinks when copying everyt…

    …hing into the "daily_gathered_plots" directory.
    
    It was meant to save space but caused problems on some platforms when trying to upload the images online.
    mmacferrin committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    13a7ed8 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. Merge pull request #16 from nsidc/operationalize-db-generation

    [WIP] Operationalize pickle "database" creation and update
    mfisher87 committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    2040671 View commit details
    Browse the repository at this point in the history
  2. Update antarctica_today/generate_daily_melt_file.py

    Co-authored-by: Matt Fisher <[email protected]>
    mfisher87 and MattF-NSIDC committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    a9384b0 View commit details
    Browse the repository at this point in the history
  3. Remove unnecessary comment

    Co-authored-by: Matt Fisher <[email protected]>
    mfisher87 and MattF-NSIDC committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    687b25c View commit details
    Browse the repository at this point in the history
  4. Merge pull request #28 from nsidc/fix_0080-netcdf_nans

    Fixed nan assertion failures
    mfisher87 committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    165ee6c View commit details
    Browse the repository at this point in the history
  5. Merge pull request #24 from nsidc/0080-netcdf

    NSIDC-0080 v2 netcdf input files
    mfisher87 committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    6c11da0 View commit details
    Browse the repository at this point in the history
  6. Merge pull request #29 from nsidc/fix_numpy_dtype_deprecation_errors

    Fix numpy dtype deprecation errors
    mfisher87 committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    5d9d1c7 View commit details
    Browse the repository at this point in the history
  7. Merge pull request #31 from nsidc/improve_data_updates_and_plot_gener…

    …ation
    
    Improve data updates and plot generation
    mfisher87 committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    3213788 View commit details
    Browse the repository at this point in the history
  8. Merge branch 'main' into v2

    mfisher87 committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    55eef90 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    98206bf View commit details
    Browse the repository at this point in the history
  10. Merge pull request #38 from nsidc/ci-workflow

    Add CI workflow (type check)
    mfisher87 committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    5db2179 View commit details
    Browse the repository at this point in the history
  11. Fix lint errors

    mfisher87 committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    606ee5e View commit details
    Browse the repository at this point in the history
  12. Add mypy to env

    mfisher87 committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    72d5fa9 View commit details
    Browse the repository at this point in the history
  13. Fix type checking errors

    mfisher87 committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    b30f1f5 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    9b6fada View commit details
    Browse the repository at this point in the history
  15. Merge pull request #39 from nsidc/lint

    Fix linting and type checking errors
    mfisher87 committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    4c4eb58 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. Configuration menu
    Copy the full SHA
    5a66795 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    18d651a View commit details
    Browse the repository at this point in the history
  3. Restore prior function interface

    Use a different type narrowing approach
    mfisher87 committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    fcfad5d View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    6ce8068 View commit details
    Browse the repository at this point in the history
  5. Merge pull request #44 from nsidc/typing-enhancements

    Typing enhancements
    mfisher87 committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    0c6f7e3 View commit details
    Browse the repository at this point in the history
  6. Re-organize operations doc

    mfisher87 committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    c5bea4a View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    35539fb View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    e79eec8 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    84e33dc View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    1972323 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    e4cb83e View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    beec098 View commit details
    Browse the repository at this point in the history
  13. Forbid print statements

    mfisher87 committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    f48122b View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. WIP Add logging

    mfisher87 committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    5e52b30 View commit details
    Browse the repository at this point in the history
  2. Replace print calls with logging

    ... using loguru
    mfisher87 committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    32264c7 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #46 from nsidc/operations

    Simplify operations and ops docs
    mfisher87 committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    47d87ca View commit details
    Browse the repository at this point in the history
  4. Merge pull request #47 from nsidc/no-print

    Replace print calls with logging
    mfisher87 committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    4df6c02 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. Configuration menu
    Copy the full SHA
    eea3a51 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2784faf View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    7e549c0 View commit details
    Browse the repository at this point in the history
  4. Merge pull request #49 from nsidc/docker

    Dockerize & continuously deliver docker images
    mfisher87 committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    579e3c0 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    b120fe2 View commit details
    Browse the repository at this point in the history
  6. Merge pull request #50 from nsidc/open-with-context-manager

    Use context managers to open files
    mfisher87 committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    bf75dc2 View commit details
    Browse the repository at this point in the history
  7. Extract dev compose file

    mfisher87 committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    ebd04e5 View commit details
    Browse the repository at this point in the history
  8. Merge pull request #51 from nsidc/dev-compose

    Compose configuration for development
    mfisher87 committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    bc8492c View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    088ec25 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    e872a4a View commit details
    Browse the repository at this point in the history
  11. Merge pull request #52 from nsidc/simplify-cli-and-ops-instructions

    Simplify cli and ops instructions
    mfisher87 committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    22a1fa1 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    a90253f View commit details
    Browse the repository at this point in the history