New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

performance and MLM fine-tuning? #17

Open

paoslaos opened this issue Feb 1, 2025 · 0 comments

paoslaos commented Feb 1, 2025

Dear authors,

congrats on a very nice project with an awesome codebase. I have 2 small questions.

I was wondering if you have looked into comparing the performance vs. https://github.com/uci-cbcl/esm-efficient ?
For fine-tuning the MLM task, do you have a hint how get started? Probably providing / adapting the linked training scrpt with a checkpoint argument would do the trick?

Thanks.

Sincerely,
P.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment