Same key used for each state during training of full rainbow

[mmcaulif]

Hi, while benchmarking my own implementation of Rainbow I had been using the dopamine full version as a reference, however I noticed that in the get_logits and get_q_values functions in line 95 and 100 respectively their is no axis mapped over the key, and later on in the train script only a single key is split for each of the forward passes.

I am just curious, if given the use of Noisy Networks in the full rainbow algorithm, is this implementation detail intentional? As it will lead to the same noisy parameters for each statement and thus a higher bias to the gradient updates. From my understanding the alternative is to split the keys per forward pass and for each input, which will introduce some additional computation. I understand this might additional bias be negligible compared to the computation saved but am interested to hear :)

Thanks for your time.