How can I fetch file with permanent name without version in name from not versioned S3 like minio

[SergeyMuha]

I have

• name: binary
  type: s3
  source:
  access_key_id: ((s3_access_key_id))
  bucket: ((s3_buckets_cli))
  endpoint: ((s3_endpoint))
  secret_access_key: ((s3_secret_access_key))
  versioned_file: bbr

I get

resource script '/opt/resource/check []' failed: exit status 1

stderr:
�[31merror finding versions: bucket is not versioned
�[0m

file is in bucket with name bbr

[lukasmrtvy]

You have to enable versioning mc version enable minio/concourse and erasure coding minio server /data{1...4} ( 4 is min )
@vito fyi

[doublespaces]

This can be bypassed by inserting a timestamp into the filename. It would be nice to simply fetch a static filename without requiring versioning on the bucket however.

[afewell]

I faced a similar problem, in reading about the s3 resource I could see that providing the regexp field in the s3 resource can be used in place of a version number by allowing the resource to identify/extract a version number from the filename. But it took me a bit to figure out how to define the regexp field properly, but as noted in the description of the regexp field, you must use a capture group in your regex so the s3 resource knows which section of your filename contains the version number. Once I setup my regexp properly, the system pulled the version number from my filename and I stopped getting the error about from s3resource.Version about the missing versionid field and was able to fetch my permanently named file with no version metadata from my minio server. Hope this helps. I should note, it seems to me like this issue relates to incorrect use of the regexp field and may reflect a user error, the documentation does correctly explain how to implement the regexp field so I think this issue may be able to be closed. Thanks!

[afewell]

Also for reference, here is an example of my s3 resource that is working for me to pull a permanently named file from an unversioned minio bucket. Note the key part is the parentheses in the regexp, as the parentheses define a capture group, which is the part of your filename that will be extracted and used as the version number by the s3 resource:

- name: opsman-product
  type: s3
  source:
    endpoint: ((minio_url.token))
    access_key_id: ((minio_access_key_id.token))
    secret_access_key: ((minio_secret_access_key.token))
    bucket: ((minio_pivnet_products_bucket.token)) 
    disable_ssl: true
    skip_ssl_verification: true
    regexp: .*ops-manager-vsphere-(.*)\.ova.*
    ```