In fintech systems, milliseconds don’t just impact performance — they directly translate into financial risk. AI-driven decisions such as fraud detection, credit scoring, and transaction authorization operate under strict latency budgets, often under 100 milliseconds. When these systems slow down, the consequences go beyond user experience: delayed decisions can increase exposure, impact compliance, and lead to measurable financial loss. This talk dives into the engineering challenges of running low-latency AI systems reliably in production. We’ll explore how to define meaningful SLOs for AI workloads, manage tail latency (p95/p99), and design resilient inference pipelines on Kubernetes. Through real-world scenarios, we’ll unpack trade-offs between model accuracy and response time, how bottlenecks emerge under scale, and what typically fails first in high-pressure environments. We’ll also cover observability patterns for AI systems — from detecting latency regressions early to maintaining consistent performance under fluctuating load. If you build or operate systems where delays are unacceptable, this session will provide practical insights to design AI platforms that are not only intelligent, but reliably fast under real-world conditions.
Anisha Manoharan is a Senior Platform Engineer specializing in cloud-native systems, AI infrastructure, and scalable platform engineering. She has extensive experience building and operating distributed systems using Kubernetes, enabling teams to move from experimental models to reliable, production-grade AI platforms. With a background in SRE and DevOps, Anisha has worked on automation, monitoring, and resilient infrastructure in regulated environments, bringing a strong focus on reliability, scalability, and real-world impact. She is passionate about bridging the gap between machine learning and production systems, sharing practical insights on how modern engineering teams can successfully deploy and scale AI in real-world applications.
