Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Additional exception handing for NTD API operator #3474

Open
charlie-costanzo opened this issue Sep 24, 2024 · 0 comments
Open

Additional exception handing for NTD API operator #3474

charlie-costanzo opened this issue Sep 24, 2024 · 0 comments
Labels
data-pipeline-ingestion-and-modeling Ingesting, parsing and modeling data. Evan Siroky is product owner.

Comments

@charlie-costanzo
Copy link
Member

charlie-costanzo commented Sep 24, 2024 •
edited
Loading

Following up on #3467 and #3472, because we wanted to get MVP tables into the warehouse, we are returning to add Exception handling for any non-200 response from the API (e.g., a 404 response will still return some JSON, but not in the structure that's valid for the table).

As captured in this PR comment

Also if desired:
Add handling for prepending test to bucket names in local airflow runs
@charlie-costanzo charlie-costanzo added this to the National Transit Database milestone Sep 24, 2024
@charlie-costanzo charlie-costanzo mentioned this issue Sep 24, 2024
Merged
5 tasks
@evansiroky evansiroky added the data-pipeline-ingestion-and-modeling Ingesting, parsing and modeling data. Evan Siroky is product owner. label Sep 24, 2024
@ohrite ohrite removed this from the National Transit Database milestone Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
data-pipeline-ingestion-and-modeling Ingesting, parsing and modeling data. Evan Siroky is product owner.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ohrite @evansiroky @charlie-costanzo