Autoscaling AI Workloads on Kubernetes With KEDA and What It Means for Agentic Systems
Cloud Native Now, Wednesday, August 26th, 2026
KEDA's queue-based autoscaling fixes a blind spot in Kubernetes HPA for bursty AI and agentic workloads.
Author Kishor Patil explains that standard Horizontal Pod Autoscaling watches CPU and memory, metrics that miss the real problem for AI inference: 'a queue can build while existing pods still show low CPU or memory utilization,' so HPA reacts only after latency and backlog worsen.
KEDA (Kubernetes Event-Driven Autoscaling) instead scales workers based on external queue depth from sources like Pub/Sub, Redis, RabbitMQ or SQS, scaling down to zero when idle.
Recommendations include tuning scaling thresholds carefully to avoid thrashing, keeping one warm replica to absorb bursts and avoid 30-second cold starts, and monitoring queue depth alongside pod count, with agentic AI workloads showing even more extreme burstiness than standard model serving.