Fine-tuning Stable Diffusion on Heroicons to Create an Infinite Icon Library

Modal fine-tuned a Stable Diffusion model on the roughly 300-SVG Heroicons library to generate new black-and-white line icons in that style, including ones for concepts like barack-obama and golden-retriever. The full pipeline — downloading the pretrained model, full fine-tuning with HuggingFace Diffusers' train_text_to_image.py script, and serving a Gradio web UI — runs on Modal's serverless GPU platform. The write-up walks through dataset prep (SVG-to-PNG conversion, white backgrounds, caption metadata.csv) and notes that full fine-tuning worked best, though LoRA-style parallel adapters can also work for small datasets.