What should be the size of the input_question in engine.py ?

[jiteshm17]

Hi, thank you so much for providing your code!
I want to check the output shapes at every stage of the model. I plan to do that by passing in some random tensors of the specific shape required model as I really cannot download and extract the entire VQA dataset just for testing.

Can anyone please let me know the shape of input_question in the engine.py file here (with and without attention) before passing it as input to the model and how to create a random tensor of that specific shape. I tried using

word_to_ix = {"hello": 0, "world": 1}
lookup_tensor = torch.tensor([word_to_ix["world"]], dtype=torch.long).cuda()

input_img = torch.randn(1,2048,14,14)
input_img = x1.long()
input_img = Variable(torch.LongTensor(input_img)).cuda()

out = model(input_img,lookup_tensor)

But I get this error

ensor([1], device='cuda:0')
Traceback (most recent call last):
  File "test.py", line 83, in <module>
    out = model(x1,lookup_tensor)
  File "/home/sarvani/anaconda3/envs/MMTOD_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/sarvani/Desktop/SaiCharan/misc/vqa.pytorch/vqa/models/att.py", line 160, in forward
    x_q_vec = self.seq2vec(input_q)
  File "/home/sarvani/anaconda3/envs/MMTOD_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/sarvani/Desktop/SaiCharan/misc/vqa.pytorch/vqa/models/seq2vec.py", line 62, in forward
    lengths = process_lengths(input)
  File "/home/sarvani/Desktop/SaiCharan/misc/vqa.pytorch/vqa/models/seq2vec.py", line 12, in process_lengths
    max_length = input.size(1)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

Any help would be appreciated. Thanks!

[Cadene]

@jiteshm17 Thanks for your interest :)

The lookup_tensor should be of size [1,2] instead of [2] because you have to consider the batch size (which is 1 in your example).

lookup_tensor = lookup_tensor.unsqueeze(0)
out = model(input_img,lookup_tensor)

Does it work?

[jiteshm17]

Thank you for your response. Sorry for the delay
It actually does not seem to work, when I am printing the shape, it is torch.Size([1, 1])
The new error after I perform unsqueeze_(0) is TypeError: iteration over a 0-d tensor

  File "test.py", line 85, in <module>
    out = model(x1,lookup_tensor)
  File "/home/sarvani/anaconda3/envs/MMTOD_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/sarvani/Desktop/SaiCharan/misc/vqa.pytorch/vqa/models/att.py", line 160, in forward
    x_q_vec = self.seq2vec(input_q)
  File "/home/sarvani/anaconda3/envs/MMTOD_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/sarvani/Desktop/SaiCharan/misc/vqa.pytorch/vqa/models/seq2vec.py", line 62, in forward
    lengths = process_lengths(input)
  File "/home/sarvani/Desktop/SaiCharan/misc/vqa.pytorch/vqa/models/seq2vec.py", line 13, in process_lengths
    lengths = list(max_length - input.data.eq(0).sum(1).squeeze())
  File "/home/sarvani/anaconda3/envs/MMTOD_env/lib/python3.7/site-packages/torch/tensor.py", line 456, in __iter__
    raise TypeError('iteration over a 0-d tensor')
TypeError: iteration over a 0-d tensor

My code is as follows

word_to_ix = {"hello": 0, "world": 1}
lookup_tensor = torch.tensor([word_to_ix["hello"]], dtype=torch.long).cuda()
lookup_tensor = lookup_tensor.unsqueeze(0)
print(lookup_tensor.shape)

x1 = torch.randn(1,2048,14,14)
x1 = x1.long()
x1 = Variable(torch.LongTensor(x1)).cuda()

out = model(x1,lookup_tensor)