ERROR :All attempts to get a Google authentication bearer token failed, returning an empty token

[DA21S321D]

I code as your tutorial says in https://waymo-research.github.io/waymax/docs/notebooks/data_demo.html . I even can't run the first two block Here is my simplest code that apprently enough to recognize:

import dataclasses
from waymax import config as _config
from waymax import dataloader

config = dataclasses.replace(_config.WOD_1_1_0_TRAINING, max_num_objects=32)
data_iter = dataloader.simulator_state_generator(config=config)
scenario = next(data_iter)

Here is error:
F:\python_envirments\waymax\Scripts\python.exe C:\Users\13420\Desktop\waymax\22.py
2023-10-31 18:46:30.457267: W tensorflow/tsl/platform/cloud/google_auth_provider.cc:184] All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "NOT_FOUND: Could not locate the credentials file.". Retrieving token from GCE failed with "FAILED_PRECONDITION: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Could not resolve host: metadata.google.internal".
Traceback (most recent call last):
File "C:\Users\13420\Desktop\waymax\22.py", line 13, in
scenario = next(data_iter)
File "C:\Users\13420\Desktop\waymax\waymax\dataloader\dataloader_utils.py", line 229, in get_data_generator
for example in dataset.as_numpy_iterator():
File "F:\python_envirments\waymax\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 4691, in next
return nest.map_structure(to_numpy, next(self._iterator))
File "F:\python_envirments\waymax\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 809, in next
return self._next_internal()
File "F:\python_envirments\waymax\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 772, in _next_internal
ret = gen_dataset_ops.iterator_get_next(
File "F:\python_envirments\waymax\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py", line 3055, in iterator_get_next
_ops.raise_from_not_ok_status(e, name)
File "F:\python_envirments\waymax\lib\site-packages\tensorflow\python\framework\ops.py", line 5888, in raise_from_not_ok_status
raise core._status_to_exception(e) from None # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.PermissionDeniedError: {{function_node _wrapped__IteratorGetNext_output_types_31_device/job:localhost/replica:0/task:0/device:CPU:0}} Error executing an HTTP request: HTTP response code 403 with body 'AccessDeniedAccess denied.

Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object. Permission 'storage.objects.get' denied on resource (or it may not exist).

'
when reading gs://waymo_open_dataset_motion_v_1_1_0/uncompressed/tf_example/training/training_tfexample.tfrecord-00000-of-01000 [Op:IteratorGetNext] name:

Process finished with exit code 1

plz help me

[DA21S321D]

by the way, my tensorflow is the latest 2.14.0

[justinjfu]

Hi @DA21S321D,

Could you try the following diagnostic steps? This would help us write an FAQ for dataset access since we're getting quite a few of these issues raised on github:

(1) Double-check that you can access WOMD via the command line

$ gsutil ls gs://waymo_open_dataset_motion_v_1_2_0/uncompressed
gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/lidar/
gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario/
gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/tf_example/

If this command fails, there is likely a problem with the gcloud login (please verify if the command gcloud auth login <your_email> worked.)

(2) Double-check that the default credentials files exist. On Linux/Mac, there should be a file called ~/.config/gcloud/application_default_credentials.json. On Windows, it should be under C:\Users\<USER-NAME>\AppData\Roaming\gcloud\application_default_credentials.json

If that file doesn't exist, you need to run gcloud auth application-default login

(3) If you are running in colab, or if the above doesn't work, you may need to manually set os.environ['GOOGLE_APPLICATION_CREDENTIALS'] =r'.../path/to/application_default_credentials.json' (see #10)

(4) If the command from step (1) worked, an alternative to running directly from Google cloud storage is to download the dataset locally.

First use gcloud to download the dataset of your choice to your local hard drive. E.g. run gsutil cp -r gs://waymo_open_dataset_motion_v_1_1_0/uncompressed/tf_example/training <YOUR FILEPATH HERE>

Next, you can manually set the dataset path on a config using dataclass's replace method. E.g. config = dataclasses.replace(config.WOMD_1_1_0_TRAINING, path=<YOUR_FILEPATH_HERE>)

Let me know if any of these work or help identify the issue.

[DA21S321D]

Hi, i am so confused that even i followed your steps, i still can't run properly. i run on colab here is my error:

i successfully login and:

[justinjfu]

Could you try the instructions on this website? https://www.tensorflow.org/datasets/gcs#simplified_instructions

In particular, add this to your colab:

from google.colab import auth
auth.authenticate_user()

[DA21S321D]

success! Thanks myman!

[cjybupt]

Could you try the instructions on this website? https://www.tensorflow.org/datasets/gcs#simplified_instructions

In particular, add this to your colab:

from google.colab import auth
auth.authenticate_user()

Yes!it works on colab.TX for solving the same problem in #28

[JinQiangWang2021]

Hi @justinjfu, I have download the data

run gsutil cp -r gs://waymo_open_dataset_motion_v_1_1_0/uncompressed/tf_example/training
Next, you can manually set the dataset path on a config using dataclass's replace method. E.g. config = dataclasses.replace(config.WOMD_1_1_0_TRAINING, path=<YOUR_FILEPATH_HERE>)

but i also met some question when I load dataset, and I hope you can give me some advice,thanks!

Traceback (most recent call last):
File "f:\CodewithPaper\waymo\main.py", line 17, in
scenario = next(data_iter)
File "C:\Users\Wang\AppData\Local\anaconda3\envs\waymo\lib\site-packages\waymax\dataloader\dataloader_utils.py", line 229, in get_data_generator
for example in dataset.as_numpy_iterator():
File "C:\Users\Wang\AppData\Local\anaconda3\envs\waymo\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py", line 4733, in next
return nest.map_structure(to_numpy, next(self._iterator))
File "C:\Users\Wang\AppData\Local\anaconda3\envs\waymo\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 810, in next
return self._next_internal()
File "C:\Users\Wang\AppData\Local\anaconda3\envs\waymo\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 773, in _next_internal
ret = gen_dataset_ops.iterator_get_next(
File "C:\Users\Wang\AppData\Local\anaconda3\envs\waymo\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py", line 3024, in iterator_get_next
_result = pywrap_tfe.TFE_Py_FastPathExecute(
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 165: invalid continuation byte