New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Is EMA used to update the training weights? #63

Open

StarCycle opened this issue Apr 19, 2024 · 0 comments

StarCycle commented Apr 19, 2024

Hello @cheng-chi ,

In your colab notebook I find that you mention the EMA model can "accelerate training and improves stability":

However, in the training loop, you may only update the EMA model with the training model:

The ema.step function will not update/smooth the training weights. Why can it accelerate training?

If my understanding is wrong, please let me know

Best,
StarCycle

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment