Expected input dimensions in first layer of ResNet50

[Maenansyn]

I have a customized model. In the forward-function I separate a (6,240,320) tensor (2 modals of images) into 2 tensors and forward them to two branches, each being a ResNet50. Each forwarded tensor has a size of (3,240,320) as planned.

But I get this Runtime Error right after it in the forward-chain:

RuntimeError: Expected 4-dimensional input for 4-dimensional weight [64, 3, 7, 7], but got 3-dimensional input of size [3, 240, 320] instead

The test_pipeline I use is shortened to this:

test_pipeline = [
    dict(type='LoadMultiModalImagesFromFiles'),
    dict(
        type='MultiScaleFlipAug',
        img_scale=(320, 240),
        transforms=[
            dict(type='MultiModalFormatBundle'),
            dict(type='Collect', keys=['img'], meta_keys=('filename', 'ori_filename', 'ori_shape',
                            'img_shape', 'pad_shape', 'scale_factor', 'img_norm_cfg')),
        ])
]

Is another one of the transforms necessary to get the requested input in the first layer of ResNet? Or what else might be the cause?

The complete error-message from the custom-forward-function onwards:

[<ipython-input-3-a1f8f2e6ef50>](https://localhost:8080/#) in forward(self, x)
     59       rgb = x[start:middle]
     60       depth = x[middle:end]
---> 61     feature_maps_colour = self.RGBNet(rgb)
     62     feature_maps_depth = self.DepthNet(depth)
     63     feature_maps_fusioned = np.maximum(feature_maps_colour, feature_maps_depth)

[/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py](https://localhost:8080/#) in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

[/content/mmdetection/mmdet/models/backbones/resnet.py](https://localhost:8080/#) in forward(self, x)
    634             x = self.stem(x)
    635         else:
--> 636             x = self.conv1(x)
    637             x = self.norm1(x)
    638             x = self.relu(x)

[/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py](https://localhost:8080/#) in _call_impl(self, *input, **kwargs)
   1049         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1050                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1051             return forward_call(*input, **kwargs)
   1052         # Do not call functions when jit is used
   1053         full_backward_hooks, non_full_backward_hooks = [], []

[/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py](https://localhost:8080/#) in forward(self, input)
    441 
    442     def forward(self, input: Tensor) -> Tensor:
--> 443         return self._conv_forward(input, self.weight, self.bias)
    444 
    445 class Conv3d(_ConvNd):

[/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py](https://localhost:8080/#) in _conv_forward(self, input, weight, bias)
    438                             _pair(0), self.dilation, self.groups)
    439         return F.conv2d(input, weight, bias, self.stride,
--> 440                         self.padding, self.dilation, self.groups)
    441 
    442     def forward(self, input: Tensor) -> Tensor:

RuntimeError: Expected 4-dimensional input for 4-dimensional weight [64, 3, 7, 7], but got 3-dimensional input of size [3, 240, 320] instead

[Maenansyn]

I think I partially found the cause - but it didn't fix the chain. I didn't knew it was still working with a batch at that point of the tool chain, which I think is the case because of similar discussions. After the last transform in the pipeline I have a (6,240,320)-tensor, but a (1,6,240,320)-tensor is stored as data in single_gpu_test (probably due to MultiScaleFlipAug?). Is the additional dimension for the batch-shape? In my own custom forward-function I squeezed this dimension (currently only testing with a single image in the cfg-file and the labels-file, therefore no actual batch, but a batch-size per gpu of 2).

But when changing it to preserve the batch-dimension I get a TypeError:

for i in range(x.size()[0]):
    rgb[i] = x[i][start:middle]
    print(rgb.size())
    depth[i] = x[i][middle:end]
    print(depth.size())

Output:

torch.Size([1, 3, 240, 320])
torch.Size([1, 3, 240, 320])

Error-message:

[/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py](https://localhost:8080/#) in _conv_forward(self, input, weight, bias)
    438                             _pair(0), self.dilation, self.groups)
    439         return F.conv2d(input, weight, bias, self.stride,
--> 440                         self.padding, self.dilation, self.groups)
    441 
    442     def forward(self, input: Tensor) -> Tensor:

TypeError: conv2d() received an invalid combination of arguments - got (list, Parameter, NoneType, tuple, tuple, tuple, int), but expected one of:
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, tuple of ints padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (!list!, !Parameter!, !NoneType!, !tuple!, !tuple!, !tuple!, int)
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, str padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (!list!, !Parameter!, !NoneType!, !tuple!, !tuple!, !tuple!, int)