There seems to be some problems in the solver A3CGcnSeq2SeqSolver.

[wangxichn]

First of all, thank you very much for sharing, I learned a lot from your code.
But during my analysis, I noticed some problems in the method "solve" of A3CGcnSeq2SeqSolver class.

In the while loop at line 70 of solver/learning/a3c_gcn_seq2seq/sover.py,
the sub_obs = encoder_obs causes the observations of the subenvironment to be repeatedly refreshed by the old state.
So sub_obs = next_sub_obs at line 90 will have no effect.
The value of next_sub_obs is not used.

It should be correct to write sub_obs = encoder_obs outside the loop to do only the first initialization.

Is my analysis correct? I see Virne also has this problem in your new project, so it's probably not a debugging error.

Thanks again for sharing the code, that's all I was wondering.