⚕️Embeddings

The textual inversion

Overview

Embedding is the result of textual inversion, a method to define new keywords in a model without modifying it. The method has gained attention because its capable of injecting new styles or objects to a model with as few as 3 -5 sample images.

How does textual inversion work?

The textual inversion is NOT the ability to add new styles or objects — other fine-tuning methods can do that as well or better. It is the fact that it can do so without changing the model.

New embedding is found for the new token S* through textual inversion. Source: original research article

First you define a new keyword that’s not in the model for the new object or style. That new keyword will get tokenized (that is represented by a number) just like any other keywords in the prompt.

Each token is then converted to a unique embedding vector to be used by the model for image generation.

Textual inversion finds the embedding vector of the new keyword that best represents the new style or object, without changing any part of the model. You can think of it as finding a way within the language model to describe the new concept.

Example of embeddings

Embedding an object

Toy cat can be used with other existing concepts (boat, backpack) in the model

Example of embedding an object.

Embedding a style

Example of embedding a style.

Where to find embeddings?

Hugging Face host the Stable Diffusion Concept Library, which is a repository of large number of custom embeddings.

Civtai is another great site you can browse models, including embeddings. Filter with textual inversion to view embeddings only.

How to use embeddings?

Web interface

Stable Diffusion Conceptualizer is a great way to try out embeddings without downloading them.

The downside of web interface is you cannot use the embedding with a different model or change any parameters.

AUTOMATIC111

First, download an embedding file from the Concept Library. It is the file named learned_embedds.bin. Make sure don’t right click and save in the below screen. That will save a webpage that it links to. Click of the file name and click the download button in the next page.

Next, rename the file as the keyword you wanted to use this embedding with. It has to be something not exist in the model. marc_allante.bin is a good choice.

Put it in the embeddings folder in the GUI’s working directory: `stable-diffusion-webui/embeddings`

Restart the GUI. In startup terminal, you should see a message like:

Loaded a total of 1 textual inversion embeddings.
Embeddings: marc_allante

Use the filename as part of the prompt to

(marc_allante:1.2)  a dog

Checking the embeddings are using in AUTOMATIC1111

There’s a button between the trash and the copy buttons:

Click it and you will see all the embeddings that are available. They are all under the Textual Inversion tab.

Clicking any of them will insert that into the prompt. This function is especially useful to eliminate the tedious work of making sure you’ve entered the embedding magic word correctly.

Pros and cons of using embedding

Pros

• It is small size (100KB or less)

Cons

• The drawback of using embedding is sometimes its not clear which model it is supposed to be used with.

Credit

How to use embeddings in Stable Diffusion - Stable Diffusion ArtStable Diffusion Art