A3C on GPU

[treksn]

Hi.
Really nice job. This is the most readable and "easiest" code I found for the A3C implementation. With regular tensorflow on CPU the code is working fine, but with tensorflow-gpu I get the error below.

Do you know, why this is happening and is it possible to get the A3C code working with GPU accelleration?
Thanks in advance!

Caused by op 'IsVariableInitialized_16/IsVariableInitialized_22/IsVariableInitialized/IsVariableInitialized_13/IsVariableInitialized_6/IsVariableInitialized_7', defined at:
  File "C:\Users\trek\.vscode\extensions\ms-python.python-2018.3.1\pythonFiles\PythonTools\visualstudio_py_debugger.py", line 2068, in new_thread_wrapper
    func(*posargs, **kwargs)
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\threading.py", line 882, in _bootstrap
    self._bootstrap_inner()
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\threading.py", line 914, in _bootstrap_inner
    self.run()
  File "d:\Thesis\Code\examples\cartpole\a3c2.py", line 159, in run
    action = self.get_action(state)
  File "d:\Thesis\Code\examples\cartpole\a3c2.py", line 209, in get_action
    policy = self.actor.predict(np.reshape(state, [1, self.state_size]))[0]
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\site-packages\keras\engine\training.py", line 1835, in predict
    verbose=verbose, steps=steps)
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\site-packages\keras\engine\training.py", line 1330, in _predict_loop
    batch_outs = f(ins_batch)
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\site-packages\keras\backend\tensorflow_backend.py", line 2476, in __call__
    session = get_session()
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\site-packages\keras\backend\tensorflow_backend.py", line 192, in get_session
    [tf.is_variable_initialized(v) for v in candidate_vars])
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\site-packages\keras\backend\tensorflow_backend.py", line 192, in <listcomp>
    [tf.is_variable_initialized(v) for v in candidate_vars])
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\variables.py", line 1203, in is_variable_initialized
    return state_ops.is_variable_initialized(variable)
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\state_ops.py", line 180, in is_variable_initialized
    return gen_state_ops.is_variable_initialized(ref=ref, name=name)
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\ops\gen_state_ops.py", line 175, in is_variable_initialized
    result = _op_def_lib.apply_op("IsVariableInitialized", ref=ref, name=name)
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 768, in apply_op
    op_def=op_def)
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 2336, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "C:\Users\trek\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 1228, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Cannot colocate nodes 'IsVariableInitialized_16/IsVariableInitialized_22/IsVariableInitialized/IsVariableInitialized_13/IsVariableInitialized_6/IsVariableInitialized_7' and 'Adam_1/iterations: Cannot merge devices with incompatible types: '/job:localhost/replica:0/task:0/device:GPU:0' and '/job:localhost/replica:0/task:0/device:CPU:0'
	 [[Node: IsVariableInitialized_16/IsVariableInitialized_22/IsVariableInitialized/IsVariableInitialized_13/IsVariableInitialized_6/IsVariableInitialized_7 = IsVariableInitialized[_class=["loc:@Adam/beta_1", "loc:@Adam/beta_2", "loc:@Adam_1/iterations", "loc:@Variable_27", "loc:@Variable_4", "loc:@dense_3/bias"], dtype=DT_FLOAT](Variable_4)]]