Release LM-Cocktail
LM-Cocktail
Merge language models (e.g., Llama, bge) to improve the general ability of models.
This method can be used to:
- Mitigate the Problem of Catastrophic Forgetting
- Improve the performance of new tasks without fine-tuning
- Approximate multitask learning or model ensemble