Extract EMA weights option

[trihardseven]

What does this do exactly? What's the benefits of using it? When should one use it?

[d8ahazard]

LOL, you tell me. :P

So, EMA weights are "Estimated Moving Average". I think this means that "non-ema" weights are like, "all" the weights the model has ever trained, where EMA is more relevant to the last bits of training that were done.

The bit of code I added that allows extracting one or the other has a disclaimer right in it that says that Non-EMA weights are better for finetuning, and EMA weights are better for inference (image generation?). So, from my understanding, we don't want to extract the EMA weights, but using them while training and saving them might be better for the final output of the newly generated model?

But, this is all just educated guessing. I added the option because someone saw the message in the log about being able to do it, not necessarily because it's better. But, if people want to try and share their findings, well, that's how the community learns and gets better at this stuff. ;)