Help with editing the NER script

[adityaarunsinghal]

Hi!

I apologize I am really new to Jiant and also Huggingface but I am on the lookout for something specific and would love some help for a class project.

What we want to do:
We have a json file with a NER dataset in the CoNLL format (MIT movie corpus). We have a json file with a QA dataset in the squAD format (moviesQA). None of these are on huggingface but they DO work with huggingface scripts if the path is provided.
We want to do some basic multi task learning with a roberta base model on MIT movie corpus + squad (which we know is on jiant) and then evaluate on MoviesQA.

The problem:
We do not want to add a task from scratch because it seems a tiny bit complicated (unless you think otherwise). Is there a way to adapt the existing ner.py script (edge probing?) to support MIT movie corpus?

Any advice would greatly help (even if it includes a completely different method of reaching this goal).

Thanks a lot!