[ Video_main.py : SSv2 dataset ]

[IemProg]

Hi,

Wonderful work, thanks for sharing the code.

I'm trying to re-train the model for SSv2 dataset using AdaptFormer. I wasn't able to the use converted checkpoint provided (videomae_pretrain_vit_b_1600.pth.)

It raises an error "Warning: Double check" (line:318, main_video.py) since it does not correspond to any if code blocks mentionned in the main_video.py.

What should be the label for this checkpoint, such that the model loads the weights and starts training ?

Thanks

[wgcban]

I got following msg with the processed model weights:
_IncompatibleKeys(missing_keys=['blocks.0.adaptmlp.down_proj.weight', 'blocks.0.adaptmlp.down_proj.bias', 'blocks.0.adaptmlp.up_proj.weight', 'blocks.0.adaptmlp.up_proj.bias', 'blocks.1.adaptmlp.down_proj.weight', 'blocks.1.adaptmlp.down_proj.bias', 'blocks.1.adaptmlp.up_proj.weight', 'blocks.1.adaptmlp.up_proj.bias', 'blocks.2.adaptmlp.down_proj.weight', 'blocks.2.adaptmlp.down_proj.bias', 'blocks.2.adaptmlp.up_proj.weight', 'blocks.2.adaptmlp.up_proj.bias', 'blocks.3.adaptmlp.down_proj.weight', 'blocks.3.adaptmlp.down_proj.bias', 'blocks.3.adaptmlp.up_proj.weight', 'blocks.3.adaptmlp.up_proj.bias', 'blocks.4.adaptmlp.down_proj.weight', 'blocks.4.adaptmlp.down_proj.bias', 'blocks.4.adaptmlp.up_proj.weight', 'blocks.4.adaptmlp.up_proj.bias', 'blocks.5.adaptmlp.down_proj.weight', 'blocks.5.adaptmlp.down_proj.bias', 'blocks.5.adaptmlp.up_proj.weight', 'blocks.5.adaptmlp.up_proj.bias', 'blocks.6.adaptmlp.down_proj.weight', 'blocks.6.adaptmlp.down_proj.bias', 'blocks.6.adaptmlp.up_proj.weight', 'blocks.6.adaptmlp.up_proj.bias', 'blocks.7.adaptmlp.down_proj.weight', 'blocks.7.adaptmlp.down_proj.bias', 'blocks.7.adaptmlp.up_proj.weight', 'blocks.7.adaptmlp.up_proj.bias', 'blocks.8.adaptmlp.down_proj.weight', 'blocks.8.adaptmlp.down_proj.bias', 'blocks.8.adaptmlp.up_proj.weight', 'blocks.8.adaptmlp.up_proj.bias', 'blocks.9.adaptmlp.down_proj.weight', 'blocks.9.adaptmlp.down_proj.bias', 'blocks.9.adaptmlp.up_proj.weight', 'blocks.9.adaptmlp.up_proj.bias', 'blocks.10.adaptmlp.down_proj.weight', 'blocks.10.adaptmlp.down_proj.bias', 'blocks.10.adaptmlp.up_proj.weight', 'blocks.10.adaptmlp.up_proj.bias', 'blocks.11.adaptmlp.down_proj.weight', 'blocks.11.adaptmlp.down_proj.bias', 'blocks.11.adaptmlp.up_proj.weight', 'blocks.11.adaptmlp.up_proj.bias', 'fc_norm.weight', 'fc_norm.bias', 'head.weight', 'head.bias'], unexpected_keys=[])