Is it possible to use a checkpoint (EN) to train another language?

[egorsmkv]

I have a language with a different symbol table. So I need to train a checkpoint from the ground or I can use already trained one?

[p0p4k]

Yes;

1. load the model with random weights and new vocab -- A
2. load the pretrained model with older vocab (and therefore different TextEncoder embedding layer). -- B
3. copy the weights from B to A in all the layers that are same size and keep others random.

[egorsmkv]

If you could add this code it would be awesome!

[egorsmkv]

@p0p4k hi! have you been working on it?

[p0p4k]

https://github.com/anonymous-pits/pits/blob/main/utils.py
You can refer that load checkpoint diff size and load checkpoint functions for reference.

[lumpidu]

@egorsmkv : so how was the outcome ? Have you tried it ?

[egorsmkv]

@egorsmkv : so how was the outcome ? Have you tried it ?

No, I haven't tried it yet but plan to