Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missleading errors when using discrete action space with CanonicalSpecWrapper #294

Open
Whojo opened this issue Mar 28, 2023 · 0 comments
Open

Comments

@Whojo
Copy link

Whojo commented Mar 28, 2023

Problem

Using an environment which uses a DiscreteArray as action's spec (e.g. gym's CartPole) with the CanonicalSpecWrapper (which, to the best of my knowledge, should be used with continuous action space only) returns the following error, which is quite missleading.

Traceback (most recent call last):
  File "run_sac.py", line 86, in <module>
    app.run(main)
  File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "run_sac.py", line 73, in main
    config = build_experiment_config()
  File "run_sac.py", line 47, in build_experiment_config
    environment_spec = specs.make_environment_spec(environment)
  File "/usr/local/lib/python3.8/dist-packages/acme/specs.py", line 46, in make_environment_spec
    actions=environment.action_spec(),
  File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/single_precision.py", line 42, in action_spec
    return _convert_spec(self._environment.action_spec())
  File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/canonical_spec.py", line 54, in action_spec
    return _convert_spec(self._environment.action_spec())
  File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/canonical_spec.py", line 68, in _convert_spec
    return tree.map_structure(_convert_single_spec, nested_spec)
  File "/usr/local/lib/python3.8/dist-packages/tree/__init__.py", line 430, in map_structure
    [func(*args) for args in zip(*map(flatten, structures))])
  File "/usr/local/lib/python3.8/dist-packages/tree/__init__.py", line 430, in <listcomp>
    [func(*args) for args in zip(*map(flatten, structures))])
  File "/usr/local/lib/python3.8/dist-packages/acme/wrappers/canonical_spec.py", line 63, in _convert_single_spec
    return spec.replace(
  File "/usr/local/lib/python3.8/dist-packages/dm_env/specs.py", line 146, in replace
    return type(self)(**all_kwargs)
TypeError: __init__() got an unexpected keyword argument 'minimum'

Expected behavior

Do not raise an error.

Why is it important to solve ?

Solution

The following change should do the work, from:
https://github.com/deepmind/acme/blob/d1e69c92000079b118b868ce9303ee6d39c4a0b6/acme/wrappers/canonical_spec.py#L62

To:

 if type(spec) == specs.BoundedArray:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant