Query string is getting modified somehow

[jeffsawatzky]

Describe the bug

I am trying to load a captcha image, and it is failing.

To Reproduce

Go to this page:
https://www.amazon.com/ap/cvf/request?arb=b3b3093e-694e-4eca-b0f2-935471c78975

If will load an amazon captcha. Inspect the url of the image. It will look something like:
https://opfcaptcha-prod.s3.amazonaws.com/506ebe7024fd4a8e85878e40a3f3cfeb.jpg?AWSAccessKeyId=AKIA5WBBRBBB27TDSHCK&Expires=1731123498&Signature=qZ0ARjdg1ISat42nhi2Xns%2BPP2g%3D

Note how Signature is qZ0ARjdg1ISat42nhi2Xns%2BPP2g%3D

Now try to load the url with aiohttp. You will get a 403 response with something like:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>SignatureDoesNotMatch</Code>
    <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
    <AWSAccessKeyId>AKIA5WBBRBBB27TDSHCK</AWSAccessKeyId>
    <StringToSign>GET\n\napplication/x-www-form-urlencoded\n1731123498\n/opfcaptcha-prod/506ebe7024fd4a8e85878e40a3f3cfeb.jpg</StringToSign>
    <SignatureProvided>qZ0ARjdg1ISat42nhi2Xns+PP2g=</SignatureProvided>
    <StringToSignBytes>47 45 54 0a 0a 61 70 70 6c 69 63 61 74 69 6f 6e 2f 78 2d 77 77 77 2d 66 6f 72 6d 2d 75 72 6c 65 6e 63 6f 64 65 64 0a 31 37 33 31 31 32 33 34 39 38 0a 2f 6f 70 66 63 61 70 74 63 68 61 2d 70 72 6f 64 2f 35 30 36 65 62 65 37 30 32 34 66 64 34 61 38 65 38 35 38 37 38 65 34 30 61 33 66 33 63 66 65 62 2e 6a 70 67</StringToSignBytes>
    <RequestId>4PY6WPMQZ4JWDY8R</RequestId>
    <HostId>s6JzAyOSv/DAWKtoy+DH2TVlgXcGMSKzoZyEr4daqsnP3FUAAHw0wNdjnTN5j2H4oy8Ogs0wmOE=</HostId>
</Error>

Note here how the SignatureProvided is:
qZ0ARjdg1ISat42nhi2Xns+PP2g=
which doesn't match
qZ0ARjdg1ISat42nhi2Xns%2BPP2g%3D
from the original request. It looks like it is encoding the %, which is breaking it.

If I take that exact same original url and load it with a browser, or Postman, or python requests, it works fine. So there is something that aiohttp is doing differently.

Expected behavior

I expect the image to load the same way it does in other libraries/browsers.

Logs/tracebacks

None

Python Version

$ python --version
3.12

aiohttp Version

$ python -m pip show aiohttp
http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: 
Author-email: 
License: Apache 2
Location: /Users/jsawatzky/Code/private/ynab-cli/.venv/lib/python3.11/site-packages
Requires: aiohappyeyeballs, aiosignal, attrs, frozenlist, multidict, yarl
Required-by: aiohttp-retry, ynab-cli

multidict Version

$ python -m pip show multidict
Name: multidict
Version: 6.1.0
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /Users/jsawatzky/Code/private/ynab-cli/.venv/lib/python3.11/site-packages
Requires: 
Required-by: aiohttp, yarl

propcache Version

$ python -m pip show propcache
Name: propcache
Version: 0.2.0
Summary: Accelerated property cache
Home-page: https://github.com/aio-libs/propcache
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache-2.0
Location: /Users/jsawatzky/Code/private/ynab-cli/.venv/lib/python3.11/site-packages
Requires: 
Required-by: yarl

yarl Version

$ python -m pip show yarl
Name: yarl
Version: 1.17.1
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache-2.0
Location: /Users/jsawatzky/Code/private/ynab-cli/.venv/lib/python3.11/site-packages
Requires: idna, multidict, propcache
Required-by: aiohttp

OS

macOS 14.7

Related component

Client

Additional context

No response

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[bdraco]

Have you tried this setting? https://docs.aiohttp.org/en/stable/client_reference.html#aiohttp.ClientSession.requote_redirect_url

Potentially related issues:
#6626
#5319

[jeffsawatzky]

@bdraco I tired and it doesn't help because there is no redirect happening. Also, looks like the setting is deprecated. It might be related to #9567 as that might be query string related.

[Dreamsorcerer]

Try a pre-encoded URL. See the note at: https://docs.aiohttp.org/en/stable/client_quickstart.html#passing-parameters-in-urls

[jeffsawatzky]

@Dreamsorcerer thanks, that seems to solve my issue. Though, that is unexpected behaviour compared to all other clients that I have used.

[Dreamsorcerer]

Yes, I'm going to propose changing this in v4, as it's now our most common reported issue.