LLMs break the assumptions traditional monitoring is built on, a service can be "up" while quietly hallucinating, drifting, or burning your token budget. This talk shows how to apply SRE discipline to AI systems i.e defining SLOs for inference quality and latency, token-level metrics, prompt tracing, and structured logging across LLM pipelines. We'll then go further with chaos engineering for AI, injecting failures into model-serving infrastructure and automating resilience experiments. Attendees will leave with a practical blueprint for making LLM systems as observable and reliable as the rest of their stack, drawn from the author's book Observability for Large Language Models (Apress, 2026).
Ankush Sharma is an AI systems architect and veteran technologist with over 22 years of experience across distributed systems, cloud infrastructure, and AI platform engineering. He is the author of Observability for Large Language Models: Site Reliability and Chaos Engineering for AI at Scale (Apress, 2026) and has led engineering teams at global technology companies (Microsoft, Citrix, startups) while contributing to open-source AI infrastructure projects. He is currently Head of Infrastructure (SRE & AI Engineering) at FloQast, a late-stage fintech startup. His work has been recognized through conference talks, patents, and developer forums. He is based in the San Francisco Bay Area.
