Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add HTTP retry handling into task SDK api.client #45121

Conversation

jscheffl
Copy link
Contributor

closes: #44355

Add HTTP retry handling to Task SDK.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (1)

task_sdk/tests/api/test_client.py:101

  • There is an extra single quote in the mounts dictionary key. It should be mounts={'http://': httpx.MockTransport(mock_handle_request)}.
mounts={'http://': httpx.MockTransport(mock_handle_request)},
@jscheffl jscheffl requested a review from amoghrajesh December 20, 2024 20:11
@jscheffl jscheffl force-pushed the feature/44355-add-retry-handling-into-task-sdk-api-client branch from bc58ae6 to 5399ace Compare December 21, 2024 14:01
Copy link
Contributor

@shubhamraj-git shubhamraj-git left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, some optimisations.

task_sdk/src/airflow/sdk/api/client.py Show resolved Hide resolved
task_sdk/tests/api/test_client.py Show resolved Hide resolved
@jscheffl jscheffl force-pushed the feature/44355-add-retry-handling-into-task-sdk-api-client branch from 5399ace to 23614f0 Compare December 22, 2024 21:29
Copy link
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, thanks Jens.
Few comments

task_sdk/src/airflow/sdk/api/client.py Outdated Show resolved Hide resolved
task_sdk/src/airflow/sdk/api/client.py Outdated Show resolved Hide resolved
task_sdk/src/airflow/sdk/api/client.py Show resolved Hide resolved
task_sdk/tests/api/test_client.py Show resolved Hide resolved
@jscheffl jscheffl force-pushed the feature/44355-add-retry-handling-into-task-sdk-api-client branch from 23614f0 to aaabd09 Compare December 23, 2024 13:44
@jscheffl jscheffl force-pushed the feature/44355-add-retry-handling-into-task-sdk-api-client branch from aaabd09 to b1878bc Compare December 26, 2024 11:31
@jscheffl jscheffl requested a review from amoghrajesh December 26, 2024 11:32
Copy link
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for addressing the comments, @jscheffl!
LGTM +1 (@ashb / @kaxil if you wanna take another look)

@jscheffl
Copy link
Contributor Author

Merging as (still) no response from Kaxil/Ash... let me know "post mortem" if some polishing is needed.

@jscheffl jscheffl merged commit 61412b3 into apache:main Dec 27, 2024
49 checks passed
jason810496 pushed a commit to jason810496/airflow that referenced this pull request Dec 28, 2024
* Add HTTP retry handling into task SDK api.client

* Add logging of call failures

* Prevent task sdk tests with LocalExecutor fail with retries

* Review feedback

* Review feedback, Adjust wording

* Correct time parameters to float

* Review Feedback
jscheffl added a commit to jscheffl/airflow that referenced this pull request Dec 28, 2024
jscheffl added a commit to jscheffl/airflow that referenced this pull request Dec 29, 2024
jscheffl added a commit to jscheffl/airflow that referenced this pull request Jan 4, 2025
jscheffl added a commit to jscheffl/airflow that referenced this pull request Jan 4, 2025
jscheffl added a commit that referenced this pull request Jan 4, 2025
* Make Edge API retries configurable

* Align implementation with TaskSDK PR #45121

* Add pytests and fix retry handling

* Fix mypy on pytests

* Update changelog

* Update changelog
LefterisXefteris pushed a commit to LefterisXefteris/airflow that referenced this pull request Jan 5, 2025
* Add HTTP retry handling into task SDK api.client

* Add logging of call failures

* Prevent task sdk tests with LocalExecutor fail with retries

* Review feedback

* Review feedback, Adjust wording

* Correct time parameters to float

* Review Feedback
LefterisXefteris pushed a commit to LefterisXefteris/airflow that referenced this pull request Jan 5, 2025
* Make Edge API retries configurable

* Align implementation with TaskSDK PR apache#45121

* Add pytests and fix retry handling

* Fix mypy on pytests

* Update changelog

* Update changelog
agupta01 pushed a commit to agupta01/airflow that referenced this pull request Jan 6, 2025
* Add HTTP retry handling into task SDK api.client

* Add logging of call failures

* Prevent task sdk tests with LocalExecutor fail with retries

* Review feedback

* Review feedback, Adjust wording

* Correct time parameters to float

* Review Feedback
agupta01 pushed a commit to agupta01/airflow that referenced this pull request Jan 6, 2025
* Make Edge API retries configurable

* Align implementation with TaskSDK PR apache#45121

* Add pytests and fix retry handling

* Fix mypy on pytests

* Update changelog

* Update changelog
HariGS-DB pushed a commit to HariGS-DB/airflow that referenced this pull request Jan 16, 2025
* Add HTTP retry handling into task SDK api.client

* Add logging of call failures

* Prevent task sdk tests with LocalExecutor fail with retries

* Review feedback

* Review feedback, Adjust wording

* Correct time parameters to float

* Review Feedback
HariGS-DB pushed a commit to HariGS-DB/airflow that referenced this pull request Jan 16, 2025
* Make Edge API retries configurable

* Align implementation with TaskSDK PR apache#45121

* Add pytests and fix retry handling

* Fix mypy on pytests

* Update changelog

* Update changelog
got686-yandex pushed a commit to got686-yandex/airflow that referenced this pull request Jan 30, 2025
* Add HTTP retry handling into task SDK api.client

* Add logging of call failures

* Prevent task sdk tests with LocalExecutor fail with retries

* Review feedback

* Review feedback, Adjust wording

* Correct time parameters to float

* Review Feedback
got686-yandex pushed a commit to got686-yandex/airflow that referenced this pull request Jan 30, 2025
* Make Edge API retries configurable

* Align implementation with TaskSDK PR apache#45121

* Add pytests and fix retry handling

* Fix mypy on pytests

* Update changelog

* Update changelog
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add HTTP retry handling into task sdk's api.client
3 participants