Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Viz EC2 Services to Lambda - Part 3a - Anomaly #582

Merged
merged 27 commits into from
Dec 12, 2023

Conversation

TylerSchrag-NOAA
Copy link
Contributor

@TylerSchrag-NOAA TylerSchrag-NOAA commented Nov 1, 2023

This PR migrates the Anomaly product from the Viz EC2 to the Viz Python Preprocessing lambda function. It broadly encapsulates the following changes:

Changes:

  • New anomaly product script in the python_preprocessing lambda function that supports both 7day and 14day configurations.
  • Removal of anomaly pipeline and product files in the source-aws_loosa library
  • A second python_preprocessing lambda function, which uses the same source code, but with more RAM (I still need to add logic to choose the correct function, based on the anomaly config. This will be much easier to do once we deploy this to TI next week, so I will do it then).

Deployment Considerations:

  • I may need to resolve merge conflicts after we merge Part 2 - Rapid Onset Flooding
  • We will need to include a ingest schema db dump when deploying to UAT.

TylerSchrag-NOAA and others added 25 commits October 4, 2023 08:37
removed HWP services from the pipelines_config.yml so that the EC2 doesn't create windows services for them to run
Copy link
Collaborator

@shawncrawley shawncrawley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've looked over everything and only came across one issue, which I documented across two comments. Once those get resolved, this should be good by me.

shawncrawley
shawncrawley previously approved these changes Nov 1, 2023
Copy link
Collaborator

@shawncrawley shawncrawley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this looks good now. Well done!

@nickchadwick-noaa nickchadwick-noaa merged commit f0c67c8 into ti Dec 12, 2023
1 check passed
@nickchadwick-noaa nickchadwick-noaa deleted the viz_ec2_services_to_lambda_part3 branch December 12, 2023 19:13
nickchadwick-noaa added a commit that referenced this pull request Feb 21, 2024
This PR migrates the Anomaly product from the Viz EC2 to the Viz Python
Preprocessing lambda function. It broadly encapsulates the following
changes:

Changes:
- New anomaly product script in the python_preprocessing lambda function
that supports both 7day and 14day configurations.
- Removal of anomaly pipeline and product files in the source-aws_loosa
library
- A second python_preprocessing lambda function, which uses the same
source code, but with more RAM (I still need to add logic to choose the
correct function, based on the anomaly config. This will be much easier
to do once we deploy this to TI next week, so I will do it then).

Deployment Considerations:
- I may need to resolve merge conflicts after we merge Part 2 - Rapid
Onset Flooding
- We will need to include a ingest schema db dump when deploying to UAT.

---------

Co-authored-by: CoreyKrewson-NOAA <[email protected]>
Co-authored-by: NickChadwick-NOAA <[email protected]>
@nickchadwick-noaa nickchadwick-noaa added this to the V2.1.5 milestone Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants