AI services can look healthy at the infrastructure layer while users are already experiencing a reliability incident. A GPU may be allocated, Kubernetes may report every pod as ready, and the model endpoint may still suffer from queueing, slow time-to-first-token, tail-latency spikes, noisy-neighbor effects, capacity exhaustion, or degraded output paths. In this talk, I’ll show how to apply SRE thinking to production AI platforms: defining meaningful SLIs and SLOs across the platform, model-serving, and application layers; deciding who owns which failure mode across SRE, platform, and ML teams; and designing shared GPU environments that can degrade gracefully instead of failing unpredictably. I’ll cover practical signals such as queue time, saturation, time-to-first-token, tail latency, successful-inference rate, and cost per successful request, along with lessons from operating Kubernetes/OpenShift AI and multi-tenant GPU workloads. The goal is to give SREs a concrete framework for answering a deceptively simple question: when an AI service is “up,” how do you know it is actually reliable?
Luca Berton is a Production AI speaker and advisor, Docker Captain, former Red Hat engineer, and author of eight technical books. He has 15+ years of enterprise infrastructure experience and speaks internationally on Kubernetes, platform engineering, AI infrastructure, automation, and reliability, including KubeCon + CloudNativeCon Europe 2026 and Red Hat Summit 2026.
