Prefacing that I'm using kohya_ss's Dreambooth LoRA to train.
To get better results for objects:
- As much as system memory allows, use as small of a batch size as possible (I used 1)
- A low learning rate (something like 0.000002) + more epochs can provide more fine-grained checkpoints in between
- Denoise the training images before using them
- Use a generic checkpoint for training (base sd1.5 or anylora works pretty well IMO)
- Try to get the object in diverse backgrounds to prevent unwanted amplification. If there's overfitting, one possible way to work around it would be to:
- generate a first-pass image
- use rembg to remove the background via an alpha channel
- convert that alpha channel into a mask
- inpaint with the mask and the original object with a different background
- Resize the images by hand to 512x512 to prevent bucketing