Support uploading log streams to Alibaba oss

[fuemoshi]

Use Cases

Alibaba's oss is compatible with the aws s3 protocol, but when uploading using the sink.aws_s3 plug-in, an error will be reported. The reason should be implementation compatibility issues. The specific configuration is as follows：

s3_test_v2:
type: "aws_s3"
region: "oss-cn-beijing"
inputs: [ "route_logs.test_a" ]
endpoint: "https://oss-cn-beijing.aliyuncs.com"
bucket: "my-bucket"
key_prefix: "test/vector/"
filename_time_format: "%Y/%m/%d/%H/1_"
auth:
access_key_id: "xxx"
secret_access_key: "xxx"
encoding:
codec: raw_message
batch:
timeout_secs: 3
buffer:
type: "disk"
max_size: 1073741824
when_full: "block"

The error content is as follows：

Response { status: StatusCode(403), headers: Headers { headers: {"server": HeaderValue { _private: H0("AliyunOSS") }, "date": HeaderValue { _private: H0("Wed, 26 Jun 2024 05:56:50 GMT") }, "content-type": HeaderValue { _private: H0("application/xml") }, "content-length": HeaderValue { _private: H0("367") }, "connection": HeaderValue { _private: H0("keep-alive") }, "x-amz-request-id": HeaderValue { _private: H0("xxxx") }, "x-oss-ec": HeaderValue { _private: H0("0003-00001401") }} }, body: SdkBody { inner: Once(Some(b"\n\n SecondLevelDomainForbidden\n Please use virtual hosted style to access.\n 667BADA26EA3F739370623E3\n oss-cn-beijing.aliyuncs.com\n 0003-00001401\n https://api.aliyun.com/troubleshoot?q=0003-00001401\n\n")), retryable: true }, extensions: Extensions { extensions_02x: Extensions, extensions_1x: Extensions } } }))

Attempted Solutions

No response

Proposal

No response

References

No response

Version

0.38.0

[jszwedko]

Thanks @fuemoshi . I think this is the same as #17351. Basically: the AWS S3 sink doesn't yet use S3's virtual hosting method of access. I'll close this one as a duplicate, but feel free to subscribe to the other issue.