My work on Multimodal Memotion Sentiment Analysis done on the Sem-Eval dataset.
I combined a pre-trained BERT and ResNext model, making some changes to model architecture and fine-tuned the resulting model on the sem-eval dataset. The inputs consisted of Memes along with their OCR texts. The task was to classify sentiments across different categories in the image.
This work was part of a larger group work with Rishabh Bajpai and Shubhad Mathur (who built their own model), and was done partly as the course project for BITS-F312 - Neural Networks and Fuzzy Logic. The final project report can be found at here.