VQA_RAC preprocessing dictionary dimension

[Liang-Di]

Hi,
After following the instruction you provided in 'QCR_PubMedCLIP' folder, we got the following error
/content/PubMedCLIP/QCR_PubMedCLIP loading dictionary from ./data/data_rad/dictionary.pkl loading DAE image data from file: ./data/data_rad/images128x128.pkl loading CLIP image data from file: ./data/data_rad/images250x250.pkl loading DAE image data from file: ./data/data_rad/images128x128.pkl loading CLIP image data from file: ./data/data_rad/images250x250.pkl Traceback (most recent call last): File "main/main.py", line 85, in <module> question_classify.load_state_dict(pretrained_model) File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1483, in load_state_dict self.__class__.__name__, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for classify_model: size mismatch for w_emb.emb.weight: copying a param with shape torch.Size([1178, 300]) from checkpoint, the shape in current model is torch.Size([1260, 300]).

And we found this bug is caused by the newly generated dictionary. The size is different from your saved_model. Could you help me with it?

[sarahESL]

Thanks for reporting this issue.

If by the saved_model you mean the type_classifier models, then you can re-run the https://github.com/sarahESL/PubMedCLIP/blob/main/QCR_PubMedCLIP/main/type_classifier.py script with respect to the datasets you want to use and save the models.

I will also update these models some time soon. But meanwhile, you can go ahead and re-train the models.

[sarahESL]

@Liang-Di Please look at the comment and discussion at #9 .