This project applies statistical physics to generative AI, exploring how concepts like entropy and the renormalization group (RG) can yield more principled models. We focus on two directions. The first is adaptive temperature scaling to maintain constant entropy across text length. Current methods violate this, degrading generation quality in longer texts. We aim to control per-token entropy as a measure of creativity vs. determinism while preserving quality. The second is applying RG ideas to design diffusion models that capture multiscale feature emergence. Natural data often show scale-invariant, power-law behavior, but current models lack such structure. Our goal is to formalize links between statistical physics and generative AI, advancing interpretability, efficiency, and robustness.

Through SuperUROP, I hope to build on my coursework and prior research by tackling a more intensive project. I am excited to deepen my understanding of generative models through physics-inspired approaches and to contribute meaningful research in this rapidly evolving field.