Feature to use Environment Variables in toml configuration file

[felixbecker]

Hi,
i hope it is ok, that I contribute a small feature which was very easy to implement and go's a long way for me.

In container environments configs gets often passed don via ENV vars.

Especially one does not want to have aws keys in the config files (I know there are other options as well implemented) Nevertheless what I did was the following:

´´´
[buckets.test.credentials]
aws_access_key_id = "aaa"
aws_secret_access_key = "bbb"

with this small change you could do something like this:

[buckets.test.credentials]
aws_access_key_id = "${ACCESS_KEY}"
aws_secret_access_key = "${SECRET_KEY}"

given `ACCESS_KEY' and 'SECRET_KEY' is defined as ENV vars it will expanded after loading the config file.

This does work with arbitrary ENV vars which are referenced in the config.

I hope you like the idea as well and like to merge this PR.

Best Regards

Felix

[moriyoshi]

You can specify aws_access_key_id and aws_secret_access_key via AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variable out of the box. Are there any other settings you find useful if placeholders could be embedded in?

[felixbecker]

Hi, thanks für your reply. I already saw the cmd flags for the aws keys. Actually in my scenario I used RoleBasedAccess via IAM Role and the ec2RoleProvider*.

But I put almost every other parameter like encryption configs and the user and password from the user accessing the server in the env vars.

Actually I assembled a Dockerfile which creates the sftp-s3-proxy.toml file on build time with the env var placeholder and the CI/CD pipeline sets the data during deployment (kubernetes secrets).
(If you interested in the Dockerfile I could also supply it via another PR)

Also the code change is small and just in addition if no env vars are set in the config.toml it will basically ignore it. Otherwise it will be replaced during the time when the config file is loaded. It opens up for flexibility especially in my deployment scenario. And you do not have to supply every conf setting as cmd flag.

But its up to you =) ...

Best Regards and thanks for sftp-s3-proxy.

• and by the way: this code is not required. This is actually standard behavior of the sdk. It basically checks for Env vars, ~/.aws/credentials, ec2RoleProvide (don't know if this is the right sequence, but it has fallbacks) - so if none of the others is set it takes the role provider.

https://github.com/moriyoshi/s3-sftp-proxy/blob/master/bucket.go#L87-L97