How to enable s3 Virtual-hosted–style access, --subdomain seems not work

[graphenn]

[root@ecs-work:~] 1 # geesefs -f --debug_s3 --subdomain --endpoint https://s3.amazonaws.com --memory-limit 700 --entry-limit 80000 example-bucket/temp/ /mnt/
2024/12/13 02:11:21.956606 s3.DEBUG HEAD https://s3.amazonaws.com/example-bucket/temp/ = 403 []
2024/12/13 02:11:21.956670 s3.DEBUG Date = [Thu, 12 Dec 2024 18:11:21 GMT]
2024/12/13 02:11:21.956673 s3.DEBUG Server = [AmazonS3]
2024/12/13 02:11:21.956676 s3.DEBUG X-Amz-Request-Id = [9319WW6Y0XYSAR7Y]
2024/12/13 02:11:21.956679 s3.DEBUG X-Amz-Id-2 = [LkYxsDEsQaWBTM5nefqZUaa1gxu3203SLt4XODaksxLKH5FZf+COgtWRBKrprb4YyA+AAZV4z8A=]
2024/12/13 02:11:21.956686 s3.DEBUG Content-Type = [application/xml]
2024/12/13 02:11:21.956948 s3.DEBUG DEBUG: Request s3/HeadObject Details:
---[ REQUEST POST-SIGN ]-----------------------------
HEAD /example-bucket/temp/uw6zhcw8ckwu185vq7rr2yirzlz8kzp3 HTTP/1.1
Host: s3.amazonaws.com
User-Agent: GeeseFS/0.42.3 (go1.22.9; linux; amd64)

I have add --subdomain, but it still use Path-style requests

[graphenn]

I found that by simply adding the --region xxxx flag, the initial HEAD request is no longer triggered, and the error is avoided. After this, the --subdomain option works as expected.

However, I also looked at an unmerged PR (kahing/goofys#403) from Goofys that addresses the issue with the first HEAD request. And discuss why we need virtual-host. Here's the relevant code snippet I modified. Instead of submitting my own PR, you can reference this change, as it works in by my test:

func (s *S3Backend) detectBucketLocationByHEAD() (err error, isAws bool) {
	u := url.URL{
		Scheme: "https",
		Host:   "s3.amazonaws.com",
		Path:   s.bucket,
	}

	if s.awsConfig.Endpoint != nil {
		endpoint, err := url.Parse(*s.awsConfig.Endpoint)
		if err != nil {
			return err, false
		}

		u.Scheme = endpoint.Scheme
		u.Host = endpoint.Host
	}

	if s.awsConfig.S3ForcePathStyle != nil && !*s.awsConfig.S3ForcePathStyle {
		bucket := strings.TrimSuffix(s.bucket, "/")

		if strings.Contains(bucket, "/") {
			parts := strings.SplitN(bucket, "/", 2)
			bucket = parts[0]
			u.Path = "/" + parts[1]
		} else {
			u.Path = ""
		}

		u.Host = fmt.Sprintf("%s.%s", bucket, u.Host)
		s3Log.Debugf("Use Virtual-hosted-style: %v", u.String())
	}

	var req *http.Request
	var resp *http.Response
        ...

The code block I added is inside the if s.awsConfig.S3ForcePathStyle != nil && !*s.awsConfig.S3ForcePathStyle block, which handles virtual-hosted style URLs. You can take this modification and adjust it as needed.

One thing I noticed, though, is that when using the --subdomain option, I can't configure a prefix like example-bucket-name/temp. This results in a 403 Forbidden error:

2024/12/13 14:47:19.319440 s3.DEBUG DEBUG: Validate Response s3/HeadObject failed, attempt 0/3, error Forbidden: Forbidden
        status code: 403, request id: 00000193BEC591F24417C7C0242EDBB5, host id: MOCKQjIxQTYxQzZDMDAwMDAxMzQwMzFCRTgwMDUyOTNBQUFBQUFBQWJiYmxxxxxx
2024/12/13 14:47:19.319471 main.FATAL Mounting file system: Unable to access 'example-bucket-name/temp': Forbidden: Forbidden
        status code: 403, request id: 00000193BEC591F24417C7C0242EDBB5, host id: MOCKQjIxQTYxQzZDMDAwMDAxMzQwMzFCRTgwMDUyOTNBQUFBQUFBQWJiYmxxxxxx

It seems like the configuration doesn't work as expected when a prefix is added under --subdomain mode.

[vitalif]

Hi, thank you for the details! I'll consider adding these changes.
Regarding initial requests, there is also the --no-detect option which disables bucket location and signature algorithm autodetection on start.