Running GRUCell on GPU

[vlasenkov]

Tried to forward a MaskedBatch through a GRUCell on GPU. Got the following:

Traceback (most recent call last):
  File "bi_gru_test.py", line 110, in <module>
    res = model(x)
  File ".../dl/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/tmp/tmp7bd5sts_/matchbox_572f.py", line 9, in forward
    matchbox.MaskedBatch, matchbox.TENSOR_TYPE)) else self.fcell(xt, hf
  File ".../dl/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File ".../dl/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 763, in forward
    self.bias_ih, self.bias_hh,
  File ".../dl/lib/python3.6/site-packages/torch/nn/_functions/rnn.py", line 54, in GRUCell
    return state(gi, gh, hidden) if b_ih is None else state(gi, gh, hidden, b_ih, b_hh)
  File ".../dl/lib/python3.6/site-packages/torch/nn/_functions/thnn/rnnFusedPointwise.py", line 24, in forward
    input_gate, hidden_gate, ibias, hbias, hx, hy, workspace)
TypeError: CudaGRUFused_updateOutput received an invalid combination of arguments - got (int, MaskedBatch, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor), but expected (int state, torch.cuda.FloatTensor input, torch.cuda.FloatTensor hidden, [torch.cuda.FloatTensor bias1 or None], [torch.cuda.FloatTensor bias2 or None], torch.cuda.FloatTensor hx, torch.cuda.FloatTensor hy, torch.cuda.FloatTensor storage)

Does it mean that matchbox requires another implementation of GRU for GPU? Is there some workarond?

[jekbradbury]

Yes, we will have to tell Matchbox that all of the rnnFusedPointwise ops are elementwise n-ary.

[vlasenkov]

Does any layer/loss that makes calls to torch._C need to have an implementation in matchbox? This results in cloning torch's API. Is it possible to create some default wrapper for all such layers? It would just apply torch._C routines to MaskedBatch.data and return a new MaskedBatch.

[jekbradbury]

Yes, but we'll have to either list them somewhere or modify PyTorch source. Eventually I definitely want to overload at the aten level but there's enough potential churn there right now with c10, and enough relevant things still implemented in python, that the current approach is likely to continue to be cleaner until PyTorch 1.0

…

On Sat, May 26, 2018 at 12:56 AM Leonid Vlasenkov ***@***.***> wrote: Does any layer/loss that makes calls to torch._C need to have an implementation in matchbox? This results in cloning torch's API. Is it possible to create some default wrapper for all such layers? It would just apply torch._C routines to MaskedBatch.data and return a new MaskedBatch. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#17 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALL4tsLWhCfsEhqaZ9wGGMUzjqSjomtJks5t2QqqgaJpZM4ULpXg> .