Access denied when providing an authentication token associated with a set of permission policies to S3FileSystem

[rub73]

I have an app with authentication done via Cognito against a user pool associated with an identity pool providing a role to authenticated users, with a set of attached policies granting access to resources such as S3 buckets.

According to the S3FileSystem documentation, passing the credentials token to a non-anon connection should cause s3sf to use the token and, thus, the policies in place. This doesn´t happen and accesses fail with a denied access error - executing the same accesses with direct boto3 apis succeeds.

[martindurant]

Could you please show how you pass your token to boto versus s3fs?

[rub73]

Sure. After authenticating, with the authentication token:

1. boto3:

• Identity id obtained by calling the get_id() method over the "cognito-identity" boto client, with the account id, identity pool id and login descriptor (( "cognito-idp..amazonaws.com/": )) as parameters, corresponding to the "IdentityId" key from the returned dict
• with the identity id, the credentials are fetched using, again, the cognito-identity client, method get_credentials_for_identity
• the returned object is finally used to define the credentials associated with a new botocore Session, encapsulated by a boto3 Session set as the boto3.DEFAULT_SESSION, from which the clients are instantiated from.

the sequence above is implemented by the streamlit_cognito_auth library, method Boto3SessionProvider.set_default_session()

2. s3sf: the token is passed as an argument to the S3SFFileSystem using the token parameter

Please let me know whether the above is clear enough

[martindurant]

In the first flow, I don't see you handle a token at all.

Note that you can pass session : aiobotocore AioSession to s3fs, if you know how to make one; and the rest should be passed via client or session kwargs. So you'll need to know what "define the credentials associated with a new botocore Session" is actually doing.