What is LORA?

[mykeehu]

I see LORA this, LORA that, but what is it? What does she know more about? Why does she need a model? Where are these models? I'm not interested scientifically, I'm interested at the user level.

@d8ahazard I know, I don't like to write help either, but I'm really getting to the point where there are so many switches in the program that I'm getting lost, and now this LORA... I'll be making coffee with DB next week, I just don't know?

New technologies are coming out every day and I'm not familiar with all of them and how they can help my training.

[mykeehu]

What I found as a starting point:

• the LORA site
• the LORA group on reddit.
• the LORA results, samples on reddit
• A detailed description by @brian6091

I'm trying to understand, but for me it's a bit high maths and how it resembles HNs. If I understand correctly, it only stores the deviations from the base model, so mixed with a base model it can be merged into a new ckpt.

[FugueSegue]

I was wondering when something like this was going to be developed. It seems like a better alternative to creating 2GB files for something like faces.

[mykeehu]

I also wondered what the difference is between TI and this, but TI only refines the model and it mostly affects the one it is trained on, while LORA seems to be stored as a difference and can be merge with other models. This is what the developer wrote professionally:
"LORA is a Dreambooth variant, not TI. LORA changes the weights of the UNet, but instead of storing them in the UNet stores them in two separate smaller matrices. X + AB; X is the UNet, A and B are the small LORA matrices."

[sashasubbbb]

Pretty new to dreambooth, and this new LORA thing might be a band-aid to all problems i disliked in dreambooth models compared to HNs, would really appreciate if someone could make up a full guide how to use LORA inside Dreambooth extension

[manniacmind]

But what do you do after a LORA model has been trained? I see 2 .pt files in the models/lora folder, but how do I use them now for creating images?

[Thomas-MMJ]

select the model you created in the top left drop down. Select the lora you want to merge in drop down menu below it. Press CKPT button at the top middle row of buttons. You will then have a new ckpt file created in your ckpt model folder.

[manniacmind]

Oh :) I was hoping the lora models could be used like textual inversion embeddings, so I could use several of them at the same time and change the base models on the fly

[Thomas-MMJ]

You can but currently the web_ui repo doesn't support it. If you are okay with a bit of coding you can just use the py notebook in the scripts folder of the original repository - https://github.com/cloneofsimo/lora#lengthy-introduction

[cloneofsimo]

Hi, @mykeehu I've notice that most of my code was copy-pasted to make LoRA work in this repo, so if you want to know what this does check my introduction to lora here: https://github.com/cloneofsimo/lora#lengthy-introduction
Also @d8ahazard, I appreciate that you've done such hard to make LoRA to get it work in A1111's repo, but just a bit of credit would be appreciated like citing to my repo in the code that used LoRA.

[mike4llison]

@d8ahazard Where do I put the LORA.pt files?