[develop] Add smoke and dust verification #1174
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…the tar file where the prepbufr files live changed"
…y Michelle Harrold, solution by Michael Kavulich.
…ntStat tasks' METplus log files.
…ing cycles for CCPA and MRMS but not yet for NDAS or NOHRSC.
…thout performing unnecessary repeated pulls.
… they're per-cycle or per-day.
…nup and comments.
…files from HPSS (and works with multiple cycles).
…e cleanup is happening.
…les, that are expected to be created once the task is finished actually get created. This is needed because it is possible that for some forecast hours for which there is overlap between cycles, the files are being retrieved and processed by the get_obs_... task for another cycle.
…nd EnsembleStat tasks such that GenEnsProd does not depend on the completion of get_obs_... tasks (because it doesn't need observations) but only forecast output while EnsembleStat does.
…d due to changes to dependencies of GenEnsProd tasks in previous commit(s).
…tending to time out for 48-hour forecasts.
…sure PcpCombine operates only on those hours unique to the cycle, i.e. for those times starting from the initial time of the cycle to just before the initial time of the next cycle. For the PcpCombine_obs task for the last cycle, allow it to operate on all hours of that cycle's forecast. This ensures that the PcpCombine tasks for the various cycles do not clobber each other's output. Accordingly, change the dependencies of downstream tasks that depend on PcpCombine obs output to make sure they include all PcpCombine_obs tasks that cover the forecast period of the that downstream task's cycle.
…ossibly also get_obs_ndas by putting in sleep commands.
…s 'ADPSFC' and 'ADPUPA' being changed to 'SFC' and 'UPA'.
…s 'ADPSFC' and 'ADPUPA' being changed to 'SFC' and 'UPA'.
…field group names 'ADPSFC' and 'ADPUPA' being changed to 'SFC' and 'UPA'.
… the names of the METplus log files.
…nistic vx. Make corresponding changes in the docs.
…asks' into feature/daily_obs_tasks
…ature/add_smoke_vx
…so, fix problems with POINT_STAT_MESSAGE_TYPE for Airnow obs
- Better description for METPLUS_VERBOSITY_LEVEL - Convert all the REMOVE_RAW_OBS_* vars to a single REMOVE_RAW_OBS_DIRS variable. Way less complicated and you should never absolutely NEED to specify this by ob type - Better descriptions for some AQ variables Includes mods to get_obs.py for REMOVE_RAW_OBS_DIRS
… add matched-pairs to stat files for smoke variables
…environment is not set
…es for bad AOD data
mkavulich
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DESCRIPTION OF CHANGES:
This PR adds verification of smoke and dust observations to the SRW verification workflow. More details to follow, including documentation.
Type of change
TESTS CONDUCTED:
DEPENDENCIES:
DOCUMENTATION:
ISSUE:
CHECKLIST
CONTRIBUTORS (optional):