some questions on variant prediction example

[vigneshvalliappan]

Hello,

Thank you for sharing your amazing work.

I have a couple of questions relating to your variant prediction example, that I hope you can help with:

1. I noticed that while a number of the sequences in your P62593.fasta file are different from each other, there are also a number of them that are identical without mutations. If you look among the first 10 sequences in the file for example, you will notice that while the labels are different, a number of the sequences themselves appear to be identical without mutations. I was under the impression that they would not be identical. Could you kindly explain if there is something I'm not understanding correctly?

2. I noticed that in addition to esm1b_t33_650M_UR50S.pt, esm1b_t33_650M_UR50S-contact-regression.pt is also downloaded when running extract.py. Could you kindly help to clarify my understanding about the following?

• Is esm1b_t33_650M_UR50S-contact-regression.pt also used in the embedding computations, beyond just esm1b_t33_650M_UR50S.pt?
• Is esm1b_t33_650M_UR50S-contact-regression.pt part of the esm-1b transformer, or does it influence the transformer in any way? My understanding is that they are separate (i.e. the transformer comes up with representations/embeddings, while contact prediction is a downstream task, where esm1b_t33_650M_UR50S-contact-regression.pt is involved). Is my understanding correct?

Thank you for your time and consideration.