Shared GPUs Without Shared Latency: Priority-Aware Serving in Red Hat AI
Red Hat, Tuesday, October 20th, 2026
Red Hat AI 3.5 adds priority-aware serving built on llm-d flow control so chat, batch and agent traffic can share GPUs.
Red Hat says chat, batch and agentic traffic can share the same GPU without interactive requests waiting on batch jobs.
Priority-aware serving, built on llm-d flow control and generally available in Red Hat AI 3.5, decides which request goes first when the pool is full, with no application changes.
On an NVIDIA H100, Red Hat reports that 95% of premium requests received their first token in under half a second while batch rejections fell from 48,224 to zero.
The post argues separate pools each pay for their own peak capacity.