LoRA Ease 🧞♂️
Train a high quality SDXL LoRA in a breeze ༄ with state-of-the-art techniques and for cheap (try out the FLUX Ease trainer)
Dreambooth with Pivotal Tuning, Prodigy and more! Use the trained LoRAs with diffusers, AUTO1111, Comfy. blog about the training script, Colab Pro, run locally or in a cloud.
Custom captioning
To improve the quality of your outputs, you can add a custom caption for each image, describing exactly what is taking place in each of them. Including TOK is mandatory. You can leave things as is if you don't want to include captioning.
Prodigy is an auto-optimizer and works good by default. If you prefer to set your own learning rates, change it to AdamW. If you don't have enough VRAM to train with AdamW, pick 8-bit Adam.
Prior preservation helps to ground the model to things that are similar to your concept. Good for faces.
Will train a textual inversion embedding together with the LoRA. Increases quality significantly. If untoggled, you can remove the special TOK token from the prompts.
Whether or not to use gradient checkpointing to save memory at the expense of slower backward pass
Scale the learning rate by the number of GPUs, gradient accumulation steps, and batch size
Your Hugging Face account doesn't have a payment method set up. Set one up here and come back here to train your LoRA
To use LoRA Ease locally with a UI, you can clone this repository (yes, HF Spaces are git repos!)
git clone https://huggingface.co/spaces/multimodalart/lora-ease
Install the dependencies in the requirements_local.txt with
pip install -r requirements_local.txt
(if you prefer, do it in a venv environment)
Now you can run LoRA Ease locally by doing a simple
python app.py
If you prefer command line, you can run our training script yourself.