Taking projects/mfmmlm.yaml for example, which run pretraining using masked frame model (MFM) and masked language model (MLM) on a single BERT:
project_dir: mfmmlm # specify the project dir for this baseline.
run_task:
- how2.yaml # run pretraining on how2 when launching `projects/taskmfmmlm.yaml`
- [vtt.yaml, vttcap.yaml, vttqa.yaml, youcook.yaml, youcookcap.yaml, crosstask.yaml, coin.yaml] # run fine-tuning tasks.
base_dir: task # a global template folder to specify each training task.
task_group:
pretrain: # section for pretraining. Most baselines differs in this section.
task_list:
- how2.yaml # reconfig `projects/task/how2.yaml`
dataset:
aligner: MFMMLMAligner # overwrite the aligner for MFMMLM training task.
model:
model_cls: MMFusionMFMMLM # overwrite the model, which constructs negative examples for MFM on-the-fly.
loss:
loss_cls: MFMMLM # overwrite the loss as MFMMLM, which combines MFM and MLM together.
fairseq: # all fairseq args can be expecified under this name.
dataset:
batch_size: 128
finetune: # section for fine-tuning tasks, we don't need to change anything here mostly since we want to see how pretraining can contribute to finetuning.
task_list: # specify the list of downstream tasks, e.g., copy `projects/task/vtt.yaml` to `projects/mfmmlm`.
- vtt.yaml
- vttqa.yaml
- youcook.yaml
- youcookcap.yaml
- crosstask.yaml
- coin.yaml
test: # section for testing.
task_list:
- test_vtt.yaml
- test_vttqa.yaml
- test_youcook.yaml
- test_youcookcap.yaml
- test_crosstask.yaml
- test_crosstask_zs.yaml
- test_coin.yaml