Notes
Written while building the fine-tuning half of Claycable, back when that was the whole product.
Low-Rank Adaptation from the maths up: why the update to a weight matrix can be tiny, what rank and alpha each control, when QLoRA is worth the quality it costs, and the hyperparameters to start from.
Hyperparameters won't rescue a bad dataset. What a training pair teaches a model, why consistency matters more than volume, and the four properties every pair needs.
Where image fine-tuning differs from text: what 15 to 50 photos need to look like, why the trigger word matters, how many steps to run for a given dataset size, and whether to use Replicate, FAL or a free Colab T4.