Alphabet tokens for MSA: "." vs "-" vs "<pad>"

[kdneuro]

I'm interested in using the MSA transformer but I'm a bit confused about different tokens in your alphabet.

There's one special token, <pad>, which I get the impression is just for end-of-sequence padding to support variable length batches, but not for intra-sequence padding (i.e. for an MSA)

Then there are two normal tokens, . and -. What is the semantic difference between these two? In an MSA, is it correct to use "-" for the alignment padding token?

Thanks in advance, and for this inspirational work.

[joshim5]

Hi Kelly, you're correct about the <pad> token.

For "gaps" in the MSA, you should use the - token. The model has never seen the . token during training, so it should not be used.

Thanks for the question and your interest!