Service Authentication using JWT

[shaileshkm]

Hi,
I have my C++ code mostly working with SDK. Thanks a bunch to Carlos to resolve issues I faced.

Now, I'm experimenting with REST APIs and I looked in SDK code for service authentication.
Here's my python code. I keep getting Invalid Credentials when I invoke REST API.
The credentials data is same as the one I used for my testing with SDK (which works fine).

import time
import jwt

now = int(time.time())
exp = now + 3600 # one hour

default_scope="https://www.googleapis.com/auth/cloud-platform"

payload = {}
payload["iss"] = client_email
###payload["sub"] = client_email # SDK does not use it
###payload["aud"] = project_id
###payload["aud"] = service_endpoint
payload["aud"] = token_uri # As per SDK
payload["iat"] = now
payload["exp"] = exp
payload["scope"] = default_scope # As per SDK

additional_headers = {}
additional_headers["kid"] = private_key_id

signed_jwt = jwt.encode(payload, private_key, headers=additional_headers, algorithm='RS256')

print signed_jwt

I then use this JWT and use following shell script

TOKEN='<JWT token>'
# List Buckets
curl -vi --insecure -H "Authorization: Bearer $TOKEN" https://storage.googleapis.com/storage/v1/b?project=proj-prototype
# List objects
curl -vi --insecure -H "Authorization: Bearer $TOKEN" https://storage.googleapis.com/storage/v1/b/basic_test3/o

Any idea, what I'm missing?
One suspicion I have is, I'm not specifying client_id anywhere. But I don't see SDK using it.

In shell script, if I replace TOKEN with an OAuth2 generated access token, then it works.
I want to make it work with service authentication.

[coryan]

Any idea, what I'm missing?

AFAIK google cloud storage (GCS) does not support self-signed JWTs, it requires an access token generated by https://oauth2.googleapis.com/token:

https://cloud.google.com/storage/docs/authentication#oauth

Note that other (all other?) Google Cloud services support self-signed JWTs.

Hope that helps.

[shaileshkm]

How does SDK do it, if you don't mind me asking.

[coryan]

Thanks Carlos.
I'm able to authenticate, get access token, and use it to access objects.

🎉

• I did not have to give project_id, client_id. So the only mandatory user-specific fields required seem to be private_key_id, private_key, client_email. Is this correct?

I think that is correct, this seems to be authoritative documentation:

https://google.aip.dev/auth/4112

• Somewhere I read max JWT expiry can be upto 24 hours. Is that correct understanding?

I do not believe that is true, I think the max expiry is 3600s, but some service accounts can be configured for longer expiry. I cannot find an authoritative answer, but this might help:

https://cloud.google.com/iam/docs/creating-short-lived-service-account-credentials#sa-credentials-oauth

PS: Just curious, why are you implementing all this yourself? Are there features missing in the C++ client library?

[shaileshkm]

One obvious reason is tests.

I'm also adding a separate implementation using libcurl.
Linking with the C++ SDK minimally requires following libraries
libgoogle_cloud_cpp_storage.a libgoogle_cloud_cpp_common.a libcrc32c.a libabsl_synchronization.a libabsl_str_format_internal.a libabsl_int128.a libabsl_time.a libabsl_time_zone.a libabsl_strings.a libabsl_strings_internal.a libabsl_throw_delegate.a libabsl_raw_logging_internal.a libabsl_bad_optional_access.a

• Besides executable size, we need to vet each library as per policy.
• Our build system is not cmake/bazel based. So I need to port build for each required library to integrate with ours. I cannot link with external libraries unless they come preinstalled with OS image.

We may end up supporting both implementations, depending on which project needs it.
For libcurl based implementation, I understand, I will lose out on possible libcurl performance optimizations, exponential backoff.

[coryan]

Besides executable size, we need to vet each library as per policy.
Our build system is not cmake/bazel based. So I need to port build for each required library to integrate with ours.

Ugh, that sounds like a lot of work, but different companies have different policies, often with good reasons.

I cannot link with external libraries unless they come preinstalled with OS image.

Would a signed binary package (.rpm, .deb or similar) from Google work? I cannot promise that we would do that, I am trying to gauge if there is enough demand to argue for it internally.

[shaileshkm]

I think that should work.
However, we are on tight timeline. I need to finish dev this month and certify within next month.

[SGrondin]

Thank you for the great code snippets, @coryan