Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Content app shouldnt retry on RA streaming #5937

Open
pedro-psb opened this issue Oct 25, 2024 · 0 comments
Open

Content app shouldnt retry on RA streaming #5937

pedro-psb opened this issue Oct 25, 2024 · 0 comments
Assignees
Labels

Comments

@pedro-psb
Copy link
Member

Our RemoteArtifact streaming uses chunked transfer, so once it has started a response, we can't fix anything anymore in case an error.

Because of that, the retry logic (any of the timeout, digest, etc) on the http downloader is just delaying a response that can't be possible right anymore.

We should add a mechanism to skip retries when calling from the chunked streaming context.

Relevant code:

  • Retries on downloader:
    retryable_errors = (
    aiohttp.ClientConnectorSSLError,
    aiohttp.ClientConnectorError,
    aiohttp.ClientOSError,
    aiohttp.ClientPayloadError,
    aiohttp.ClientResponseError,
    aiohttp.ServerDisconnectedError,
    TimeoutError,
    TimeoutException,
    DigestValidationError,
    SizeValidationError,
    )
    async with self.semaphore:
    @backoff.on_exception(
    backoff.expo,
    retryable_errors,
    max_tries=self.max_retries + 1,
    giveup=http_giveup_handler,
    )
  • Download call from stream_remote_artifact:
    downloader = remote.get_downloader(
    remote_artifact=remote_artifact, headers_ready_callback=handle_response_headers
    )
    original_handle_data = downloader.handle_data
    downloader.handle_data = handle_data
    original_finalize = downloader.finalize
    downloader.finalize = finalize
    download_result = await downloader.run()

Context

Identified this when investigating #5012 and #5725

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants