Golang AWS SDK v2 SignatureDoesNotMatch

[frostbyte73]

Bug report

• [✔] I confirm this is a bug with Supabase, not with my own application.
• [✔] I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

All operations fail with SignatureDoesNotMatch when using aws-sdk-go v2.
The same requests work with s3, minIO, and other s3-compatible providers.

To Reproduce

Steps to reproduce the behavior, please provide code snippets or a repository:

import (
	"context"
	"os"
	"testing"

	"github.com/aws/aws-sdk-go-v2/aws"
	awsConfig "github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/credentials"
	"github.com/aws/aws-sdk-go-v2/feature/s3/manager"
	"github.com/aws/aws-sdk-go-v2/service/s3"
	"github.com/stretchr/testify/require"
)

const (
	key      = "***"
	secret   = "***"
	endpoint = "https://***.supabase.co/storage/v1/s3"
	region   = "us-west-1"
	bucket   = "***"

	filepath = "/***/playlist.m3u8"
	filename = "playlist.m3u8"
)

func TestSupabase(t *testing.T) {
	opts := func(o *awsConfig.LoadOptions) error {
		o.Region = region
		o.Credentials = credentials.StaticCredentialsProvider{
			Value: aws.Credentials{
				AccessKeyID:     key,
				SecretAccessKey: secret,
			},
		}
		return nil
	}

	awsConf, err := awsConfig.LoadDefaultConfig(context.Background(), opts)
	require.NoError(t, err)

	client := s3.NewFromConfig(awsConf, func(o *s3.Options) {
		o.BaseEndpoint = aws.String(endpoint)
		o.UsePathStyle = true
		o.ClientLogMode = aws.LogRequest | aws.LogResponse | aws.LogRetries
	})

	file, err := os.Open(filepath)
	require.NoError(t, err)
	defer file.Close()

	_, err = manager.NewUploader(client).Upload(context.Background(), &s3.PutObjectInput{
		Body:        file,
		Bucket:      aws.String(bucket),
		ContentType: aws.String("application/x-mpegurl"),
		Key:         aws.String(filename),
	})
	require.NoError(t, err)
}

=== RUN   TestSupabase
SDK 2024/10/11 11:46:28 DEBUG Request
PUT /storage/v1/s3/***/playlist.m3u8?x-id=PutObject HTTP/1.1
Host: ***.supabase.co
User-Agent: m/E,G aws-sdk-go-v2/1.31.0 os/macos lang/go#1.22.1 md/GOOS#darwin md/GOARCH#arm64 api/s3#1.63.3 ft/s3-transfer
Content-Length: 452
Accept-Encoding: identity
Amz-Sdk-Invocation-Id: 37c0e7fb-2462-461f-b182-b67d59e9faa8
Amz-Sdk-Request: attempt=1; max=3
Authorization: AWS4-HMAC-SHA256 Credential=***/20241011/us-west-1/s3/aws4_request, SignedHeaders=accept-encoding;amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=***
Content-Type: application/x-mpegurl
X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
X-Amz-Date: 20241011T164628Z

SDK 2024/10/11 11:46:28 DEBUG Response
HTTP/2.0 403 Forbidden
Content-Length: 303
Access-Control-Allow-Origin: *
Alt-Svc: h3=":443"; ma=86400
Cf-Cache-Status: DYNAMIC
Cf-Ray: 8d105394e92022fd-ORD
Content-Type: application/xml; charset=utf-8
Date: Fri, 11 Oct 2024 16:46:28 GMT
Sb-Gateway-Mode: direct
Sb-Gateway-Version: 1
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: Accept-Encoding

SDK 2024/10/11 11:46:28 DEBUG request failed with unretryable error https response error StatusCode: 403, RequestID: , HostID: , api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
    s3_test.go:58:
        	Error Trace:	/***/supabase_test.go:58
        	Error:      	Received unexpected error:
        	            	operation error S3: PutObject, https response error StatusCode: 403, RequestID: , HostID: , api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
        	Test:       	TestSupabase
--- FAIL: TestSupabase (0.48s)

Expected behavior

Upload should succeed.

System information

• OS: macOS
• Go: 1.22.1

Additional context

I have confirmed that this works with aws-sdk-go v1, and there are no issues with my credentials.

[mercuryyy]

This is a big issue for us as well, we have had to stop service for some clients and move to AWS S3

[fenos]

Hey @frostbyte73 thanks for reporting the issue,
it is strange that it works with V1, I will give it a try and come back to you 👍

[realChesta]

+1, this is an issue for us since we are dependent on https://github.com/livekit/egress which seems to use the same library (aws-sdk-go v2) as well.

[municola]

Same here. This is a big issue for us aswell.

[mercuryyy]

the is still a big issue for us, @encima any word on this ?

[encima]

@mercuryyy have asked the storage team and will respond here with any updates

[fenos]

Let's continue the thread here:

#577