> For the complete documentation index, see [llms.txt](https://aisuko.gitbook.io/wiki/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://aisuko.gitbook.io/wiki/ai-techniques/stable-diffusion/lora-stable-diffusion.md). # LoRA(Stable Diffusion) ## Overview [LoRA(Low-Rank Adaptation)](/wiki/ai-techniques/adaptation/lora.md) models are small Stable Diffusion models that apply tiny changes to standard checkpoint models. They are usually 10 to 100 times smaller than checkpoint models. That makes them very attractive to people having an extensive collection of models. ## Why we need it? It offers a good trade-off between file size and training power. For example, Dreambooth is powerful but results in large model files (2-7 GBs). [Textual inversions](/wiki/ai-techniques/stable-diffusion/embeddings.md) are tiny (about 100 KBs), but you can't do as much. LoRA sits in between: Their files sizes are manageable (2-200 MBs), and the training power is decent. ***LoRA is an excellent solution to the storage problem. Like textual inversion, you cannot use a LoRA model alone. It must be used with a model checkpoint file. LoRA modifiles styles by applying small changes to the accompanying model file.*** ## How doe LoRA work on Stable Diffusion models? LoRA applies small changes to the most critical part of Stable Diffusion models: ### The cross attention layers It is the part of the model where the [image and the prompt meet](/wiki/ai-techniques/stable-diffusion/conditioning.md#feeding-embeddings-to-noise-predictor). According to the pager, it sufficient to fine-tune this part of the model to achieve good training. The [cross-attention layers ](https://aisuko.gitbook.io/wiki/ai-techniques/stable-diffusion/conditioning#cross-attention)are the yellow parts in the Stable Diffusion architecture below.

LORA fine-tunes the cross-attention layers (the QKV parts of the U-Net noise predictor). (Figure from Stable Diffusion paper.)

The weights of a cross-attention layer are arranged in **matrices. A LoRA model fine-tunes a model by adding its weights to these matrices.** ## Tips ### LoRA model needs the following phrase in the prompt \ #### filename It is the file name of the LoRA model, excluding the extension (.pt, .bin.etc) #### multiplier It is the weight applied to the LoRA model. The defaults is 1. Setting it to 0 disables the model. ### How to make sure the filename is correct? Here we can use the model button

Source: Stable Diffusion Art

Click on the Lora tab. You should see a list of LoRA models installed. Click on the one you want to use. And the LoRA phrase will be inserted in the prompt.

Source: Stable Diffusion Art

### Adjust the multiplier You may **adjust the multiplier** to crank up or tune down the effect. Setting the multiplier 0 disables the LoRA model. You can adjust the style effect between 0 and 1. ### Adjust for Dreambooth Some LoRA models are trained with Dreambooth. You will need to include a **trigger keyword** to use the LoRA model. You can find the trigger keyword on the model’s page. ### Using multiple models at same time Similar to [embeddings](/wiki/ai-techniques/stable-diffusion/embeddings.md), you can use **multiple LoRA models** at the same time. You can also use them with embeddings. ### Add the LoRA to the prompt In AUTOMATIC1111, the LoRA phrase is not part of the prompt. It will be removed after the LoRA model is applied. That means you cannot use [prompt syntax](/wiki/ai-techniques/chain/langchain/prompts.md) like \[keyword1:keyword2: 0.8] with them. ## Credit {% embed url="" %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://aisuko.gitbook.io/wiki/ai-techniques/stable-diffusion/lora-stable-diffusion.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.