⚕️Embeddings
The textual inversion
Overview
Embedding is the result of textual inversion, a method to define new keywords in a model without modifying it. The method has gained attention because its capable of injecting new styles or objects to a model with as few as 3 -5 sample images.
How does textual inversion work?
The textual inversion is NOT the ability to add new styles or objects — other fine-tuning methods can do that as well or better. It is the fact that it can do so without changing the model.
First you define a new keyword that’s not in the model for the new object or style. That new keyword will get tokenized (that is represented by a number) just like any other keywords in the prompt.
Each token is then converted to a unique embedding vector to be used by the model for image generation.
Textual inversion finds the embedding vector of the new keyword that best represents the new style or object, without changing any part of the model. You can think of it as finding a way within the language model to describe the new concept.
Example of embeddings
Embedding an object
Toy cat can be used with other existing concepts (boat, backpack) in the model
Embedding a style
Where to find embeddings?
Hugging Face host the Stable Diffusion Concept Library, which is a repository of large number of custom embeddings.
Civtai is another great site you can browse models, including embeddings. Filter with textual inversion to view embeddings only.
How to use embeddings?
Web interface
Stable Diffusion Conceptualizer is a great way to try out embeddings without downloading them.
The downside of web interface is you cannot use the embedding with a different model or change any parameters.
AUTOMATIC111
First, download an embedding file from the Concept Library. It is the file named learned_embedds.bin. Make sure don’t right click and save in the below screen. That will save a webpage that it links to. Click of the file name and click the download button in the next page.
Next, rename the file as the keyword you wanted to use this embedding with. It has to be something not exist in the model. marc_allante.bin is a good choice.
Put it in the embeddings folder in the GUI’s working directory: `stable-diffusion-webui/embeddings`
Restart the GUI. In startup terminal, you should see a message like:
Use the filename as part of the prompt to
Checking the embeddings are using in AUTOMATIC1111
There’s a button between the trash and the copy buttons:
Click it and you will see all the embeddings that are available. They are all under the Textual Inversion tab.
Clicking any of them will insert that into the prompt. This function is especially useful to eliminate the tedious work of making sure you’ve entered the embedding magic word correctly.
Pros and cons of using embedding
Pros
It is small size (100KB or less)
Cons
The drawback of using embedding is sometimes its not clear which model it is supposed to be used with.
Credit
Last updated
