Tag

#time to first token inference

Browsing posts matching the tag #time-to-first-token-inference

Jul 17, 2026 • 7 min read

354ms to First Audio: Gemma 4 31B on LiveKit Changes the Math for Voice AI

LiveKit has launched Gemma 4 31B on its Inference platform, delivering 354ms time to first audio and 192ms time to first token — while beating GPT-4.1 on agentic tool-use benchmarks. For developers building real-time voice agents, the numbers are hard to ignore.

Abhinav Ramaswamy