Question about ContextNet models. Have you tried creating a large model and comparing it to the Conformer?

[Slava715]

Hello!
Question about ContextNet models. Have you tried creating a large model and comparing it to the Conformer?
What is the best way to change the configuration of the ContextNet model to recognize noisy sound? Increase the number of layers or their size?

[titu1994]

ContextNet and Conformers of similar sizes perform about the same. There may be a 0.2 % difference but it's negligible. You can train with noise augmentation and rir samples to improve robustness of these models.

[Slava715]

The article writes about significantly better results on noisy data.
https://arxiv.org/pdf/2005.03191.pdf
Now I'm training a small ContextNet 40M model, I'll write about the results later.

[Slava715]

Yes, you are rigth between the ContextNet and Conformer models, there is practically no difference. ContextNet is faster and Conformer is more accurate.