SREday is a worldwide series of community events for engineers who build, ship and run modern software systems. Across cities around the world, we bring together people working in reliability, cloud, DevOps, observability and production engineering to share real-world experience, connect with their local community and explore how these disciplines are evolving in the age of AI.
Companies presenting:
AJ Meyers, Carlson BCM, Dalton AI, Experian, Gatling, GEICO, Glitnir Ticketing, Grainger, Guild Systems, Imubit, Incredibuild, Mirror Panel, Modal, Nadir, Nibbana, Qovery, Solo.io, The New York Times, TheRobotAge, Tiger Data, WWT
Every enterprise has a brownfield morass of legacy software that isn't exciting anymore, but still runs in production. These workloads are often business-critical, but receive little investment outside of what it takes to keep the lights on. As SREs, these workloads can be particularly annoying - legacy platforms and code without instrumentation often mean our only source of observability data comes from logs. In this talk, I'll show you how you can transparently use cloud-native tricks to instrument an application without touching the application's code, whether it runs on a VM, a container, or elsewhere. Then, I'll show you how the data collected can be used strategically to generate the four golden signals of observability that we as SREs leverage to keep production online. If you're managing the morass of legacy software in production, this talk is for you.... Read more
AI has dramatically reduced the complexity of starting a new code project. Today, you don't read the docs first - you ask your coding agent. In this talk, we'll show how this applies to performance testing. How developers with zero Gatling experience can write realistic load tests in minutes, using AI as their entry point instead of documentation, and why this changes how teams adopt performance testing altogether.... Read more
Agents need to be isolated for what they can and can't access (files/folders, production envs, LLMs, MCP tools, runtimes, etc. - think autonomous and standard Agents) and at the same time, scale up and down based on load, if/how the Agent is being used, and how it's being used. By default, Agents have access to every file, LLM, command, and authN/Z implementation that you do. At the same time, you're running Agents in a scalable, performant, and reliable way with a proven out orchestration/clustering platform. Example: You want to run 200+ Agents, but only spread across 6-8 k8s Pods. The Agents still perform as well as expected while keeping the underlying infrastructure costs (both resources and actual financial costs) low. In this session, you will learn about the OSS project that makes this happen - Agent Substrate, with key topics including: 1. Build on top of k8s primitives (clustering of resources for performant sandboxed Agents). 2. Implement a specialized and efficient a Control/Management Plane that's built for the world of agentic. 3. How Substrate Actors running Agents are 90% more efficient (proven in benchmarks) for cost optimization in comparison to Agents in k8s Pods. 4. Efficient and much smaller footprints for running Agents via Actors.... Read more
With more and more tools abstracting from a developer's workflows, understanding how git visualization tools help - not simply using it - is more important than ever. In this talk, we take a look at the history of git workflows, the basics of git, and the renaissance of understanding version control in a world gone mad for “vibe coding”.... Read more
A cheaper model does not always mean a cheaper run. Routing between models, trimming context, splitting work across agents, or adding tools and skills mid-run can look like smart optimizations. But when those changes disrupt prompt-cache reuse, they can turn discounted, reusable context into input you pay to process again—undermining the savings you expected. OpenAI Developers This talk explores why prompt caching deserves to be an architectural consideration, not an afterthought, in production LLM systems. We’ll examine how caching interacts with model routing, context management, subagents, and dynamic tool loading. Through concrete examples, we’ll compare preserving a warm cache with starting fresh, explore when breaking the cache is worth the trade-off, and show how to make these decisions based on the economics of an entire run rather than the price of an individual request. Attendees will leave with practical patterns for cache-aware workflows and an observability checklist covering cache reuse, cache-read and write costs, latency, and cost per successfully completed task. The takeaway: don’t optimize away your biggest savings. Optimize the whole run—not just the token count.... Read more
Over the past three years, I've collaborated with hundreds of SRE teams as they've integrated AI into their workflows. What has truly evolved - and what remains the same? This talk will provide practical insights and observations from the forefront of AI adoption in infrastructure. I'll discuss: The evolving skill set: What SREs need to learn today and which skills stand the test of time AI in incident management: Identifying where AI agents add value, where they fall short, and how to distinguish between the two AI SRE: What's real, and what's just hype... Read more
Claude Code, Codex, and other coding agents are becoming long-running, tool-using systems that read repositories, execute commands, modify code, call external services, and make decisions with limited human supervision. Yet most teams observe them primarily through token counts and final pull requests. This talk applies familiar SRE practices to agentic development. We’ll instrument a coding-agent workflow with OpenTelemetry, examine traces of tool calls, failures, retries, tests and code changes, and look at what those traces reveal about reliability that the final output hides. We’ll cover practical failure patterns, useful signals, and where traditional observability concepts translate well - and where they don’t.... Read more
Moving a Lambda into a service looks straightforward: wrap the handler in an API or a queue consumer and deploy. That’s what we did at first when we migrated seven Python data-ingestion Lambdas to k8s, and the early tests passed, including concurrent deliveries. As we moved more customers over and left the services running for days, code that had worked on Lambda for years started breaking. Liveness probes timed out, pods kept running out of memory during DLQ replay, tmp files collided, connections dropped and credentials expired. I’ve organised the talk around four promises we relied on by using Lambda: blocking was free, every delivery had the place to itself, nothing lived long enough to go bad, and the outside world saw a different you. We relied both on Lambda’s documented behaviour and on what we’d observed while running our application there. Moving to services forced us to revisit how we managed concurrency, connections and credentials once deliveries shared resources and processes lived longer. I’ll use evidence from the migration to explain each promise, including how fixing the blocking calls made the memory problem easier to trigger. I’ll also cover what to test alongside a rollout, including replay, sustained load and credential rotation. I’ll finish with a checklist for identifying what your application depends on its runtime to do, and what you’ll need to account for when that runtime changes.... Read more
As engineering teams accelerate software delivery with AI, documentation struggles to keep pace. Outdated runbooks, API references, and technical guides can create operational risks, slow incident response, and undermine platform reliability. What if documentation could evolve alongside the code itself? This talk explores how AI agents can detect documentation drift, identify changes in code and infrastructure, and automatically propose updates as part of the engineering workflow. We'll examine integrations with tools like Jira, Confluence, and GitHub through Model Context Protocol (MCP), along with automated validation, CI/CD integration, human approval, and governance. Through a practical example, attendees will learn how to approach documentation as a continuously maintained, verifiable component of software delivery rather than an afterthought.... Read more
AI agents can write migrations, but production changes need guardrails. This live demonstration walks through an agent-assisted PostgreSQL workflow using Tiger Cloud and the Tiger CLI: provisioning an isolated development service, generating versioned migrations, and testing correctness and performance before human-approved production deployment. We’ll cover scoped access, automated checks, lock and rewrite detection, deployment monitoring, and rollback criteria. Attendees will leave with a practical blueprint for accelerating database delivery without giving agents unchecked authority over production.... Read more
Growing alerts, new sources, growing volumes of telemetry. This talk shows an SRE Coworker that is always-on to correlate and detect subtle anomalies across sources, investigate complex hypotheses, and surfaces root causes automatically. Attend this session to discover how your team can wake up to zero alerts and fully explained incidents while you sleep.... Read more
17:00
Wrap up
Scan each other's QR codes & head to a nearby pub!
As AI agents move from experimentation into production, they are increasingly capable of executing commands, accessing enterprise systems, modifying infrastructure, and making operational decisions without continuous human supervision. But what happens when an autonomous agent makes the wrong decision at 2 AM? Who detects it, who has authority to intervene, and how do we stop the action before it becomes a production incident? Traditional SRE practices provide powerful mechanisms for observability, reliability, incident response, and recovery. Autonomous AI introduces additional challenges involving agent identity, dynamic authorization, decision accountability, and runtime enforcement. This session introduces a practical architecture for bringing AI governance directly into production operations. We'll explore how OpenTelemetry, agent registries, identity-aware policies, authorization gateways, human approval workflows, runtime monitoring, and automated kill switches can work together to make autonomous systems safer and more reliable. Through a realistic production incident scenario, we'll examine how an AI agent's proposed action can be evaluated, authorized, blocked, escalated, and documented before unintended consequences occur. Attendees will leave with actionable architectural patterns for connecting SRE practices with enforceable AI governance. The central message: Observing an AI agent is not the same as controlling it. Reliable autonomous systems require both.... Read more
You can observe every component of an enterprise and still not understand the health of the enterprise. SRE practice tells us to set objectives around what users care about, but those objectives rarely connect to a governed account of what the enterprise actually does. The missing object isn’t another metric, dashboard or SLO. It’s the business capability: the enduring definition of what the enterprise must be able to do, independent of the systems that deliver it. This talk sets out how a capability-based topology, running from business capability through applications and data to infrastructure, can become the organizing foundation for service health, semantic analysis and AI-driven diagnostics. It compares three approaches: adding capabilities to existing observability tools, making the capability model the organizing structure for service health, and using AI agents to engineer observability against topologies that change constantly. The argument throughout is that capabilities are the layer worth designing for, because they remain stable as the technology beneath them changes.... Read more
Our most critical upstream is a decades-old herd-management program on a farm-office PC. One day our dashboard showed a fraction of a farm's real milking herd. Every service was healthy, every job exited 0, and no alert fired. We heard about it from the farm, not from our monitoring. Those numbers feed emissions reporting, so a wrong count is a wrong carbon claim. We built the platform largely with AI coding agents. I will walk through incidents where every component was correct and the system was still wrong: a history purge that looked like cows leaving, one agent uploading several farms' data, a concurrency guard that swallowed updates, and "idempotent" migrations that never converged. Finally, I will show the AI agent we're layering on our alerting jobs to flag these failures as they happen and learn from what it misses, and close with a checklist for finding silent failures in your own pipelines.... Read more
AI agents now run thousands of ephemeral coding tasks a day, and each one needs a clean, fast, trustworthy environment to execute in. This talk shares hands-on lessons from building secure execution infrastructure for AI workloads at Incredibuild and islo.dev, covering CI runners, sandboxes, snapshots, caching, traces, and reproducible environments. It walks through a real sandbox provider for ephemeral task isolation, presented at an Apache Airflow Monthly Town Hall, and a meta-harness loop that took a coding agent from 0 of 5 to 5 of 5 passing tasks in about two seconds across four proposer steps. Drawing on earlier container security work at Twistlock, the talk covers what breaks when isolation, caching, and observability have to work together at CI speed.... Read more
Everything breaks. Bad deploys, traffic spikes, cloud outages, eventually some component will go down. When architecting a system, our instinct can be to try and eliminate any possible impact. But this can add complexity that paradoxically makes failures more likely, or make recovery harder. Let's look at an alternative: embracing degradation to cushion against a total outage. We'll look at a real world example of multi-region routing, explore a catalogue of failure modes and trade-offs that can blunt them, and highlight when compromise isn't an option. Fail safe without having to be failure-proof!... Read more
While observability is the first step towards building observable systems, monitoring is what enables us to action on the telemetry collected in a more automated way. Previous approaches to monitoring have relied primarily on infrastructure and service metrics, but modern approaches have embraced the idea of monitoring based on metrics that reflect the user experiences that consider business impact more accurately. In this, we're enabled to drive clarity over reliability targets, effectively prioritize reliability improvements, and hold ourselves accountability to customers. This talk will elaborate on those differences and review traditional and new approaches to monitoring, including the role of Service Level Objectives (SLOs). Additionally, we’ll cover the following: 1. Observability vs Monitoring 2. Metrics and the types of Metrics & Monitors 3. The benefits of an SLO-based approach to monitoring 4. Effective Accountability with Error Budget Policies... Read more
You build it, you run it. Fifteen years of SRE gospel, and almost nobody questions it. The rule was never about writing code. It was about a loop. Whoever wrote it understands it, so whoever wrote it carries the pager. Understanding was the point. The pager was just enforcement. Agents cut that loop. The code still ships. The author is gone. I take the assumption apart, one joint at a time. No human author, so who holds the pager. Nobody who can explain it, so the author cannot answer the 3am question. Review cannot keep up with the output, so your gate is a rubber stamp. Then the fixes that do not work, and why each one folds. Human in the loop. Make the agent better. Keep agent code out of prod. Then the reframe. If writing it no longer means understanding it, reliability has to come from somewhere else. Operability stops being something you hope the author cared about, and becomes something you check for. One real incident carries the argument. A single S3 copy call, a 5 GiB limit nobody checked, a migration that never sealed, and a cluster crash-looping while the log line said migrating objects: . None of it was hard. Nobody could see it. The principle is not dead. The assumption under it is.... Read more
New to TPUs? This talk starts with what a Cloud TPU is, how it differs from a GPU, and how to get one on Google Cloud. Then it moves to a live demo: an MCP server that lets an AI agent find TPU capacity, create a TPU VM, and start vLLM on it. We send the server a few requests, watch it answer, and tear everything down so nothing keeps billing. You leave with the steps to run your first model on a TPU.... Read more
Modern observability platforms make it easy to collect browser telemetry. The real challenge is deciding what not to collect. At Grainger, our digital platforms process approximately 1.5 billion requests per day across multiple brands, countries, and customer-facing applications. As our Real User Monitoring (RUM) adoption expanded, we discovered that a significant portion of our telemetry was generated by bots, synthetic monitoring solutions, automated testing platforms, internal tooling, and duplicate browser events. While these events increased observability costs, they provided little value when understanding real customer experiences. This session shares our journey modernizing browser observability during a Datadog RUM migration and transforming RUM from a "collect everything" model into a scalable, business-focused observability platform. Attendees will learn how we: • Identified and filtered bot, synthetic, and automated traffic before it reached our observability platform • Reduced browser event ingestion by nearly 80% (7.81 million to 1.67 million events) while maintaining visibility into real user experiences • Improved session accuracy by addressing split-session behavior, cookie inconsistencies, and telemetry fragmentation • Implemented intelligent filtering across edge, application, and browser layers to reduce noise and improve signal quality • Balanced engineering observability, business analytics, compliance requirements, and platform costs • Established a repeatable framework for determining which browser telemetry delivers operational value and which should be excluded Beyond the technical implementation, this talk explores the broader challenge facing modern SRE and observability teams: collecting the right data rather than simply collecting more data. The result was a cleaner signal, faster incident investigations, reduced observability spend, and greater confidence that our dashboards, alerts, and analytics reflect real customer behavior—not automated traffic. Whether you use Datadog, OpenTelemetry, New Relic, Dynatrace, Elastic, or another observability platform, you'll leave with practical techniques and architectural patterns to navigate RUM without limits.... Read more
We keep spare infrastructure running for the worst day of the year. It consumes resources every other day. We wanted to know how much standby capacity we actually need to buy the recovery objective we promise. We built the same Kubernetes service with three disaster recovery strategies: pilot light, warm standby, and hot standby. Then we failed the primary region and measured what happened. The tradeoff was visible on both sides. Keeping more capacity ready reduced recovery time, but increased energy use while waiting for an incident. Keeping less capacity idle saved energy, but made recovery depend on provisioning, autoscaling, cache warming, and downstream capacity exactly when the system was already under stress. We'll compare each strategy using recovery time, p99 latency, failed requests, error budget burn, and workload energy measured during both idle and recovery periods. We will also look at where scaling from a smaller standby stops being fast enough to satisfy the RTO. We'll finish with a practical method for sizing standby capacity from an RTO instead of assuming that more redundancy is always better.... Read more
16:30
Wrap up
Scan each other's QR codes & head to a nearby pub!
The New York Times Building
5620 8th Avenue, 45th Floor,
New York, NY 10018, United States
Sponsors & Partners
Want to become a sponsor? Get in touch!
Tyler Schade
GEICO
Observing the un-observable: instrumentation for your legacy workloads
Abstract
Every enterprise has a brownfield morass of legacy software that isn't exciting anymore, but still runs in production. These workloads are often business-critical, but receive little investment outside of what it takes to keep the lights on. As SREs, these workloads can be particularly annoying - legacy platforms and code without instrumentation often mean our only source of observability data comes from logs. In this talk, I'll show you how you can transparently use cloud-native tricks to instrument an application without touching the application's code, whether it runs on a VM, a container, or elsewhere. Then, I'll show you how the data collected can be used strategically to generate the four golden signals of observability that we as SREs leverage to keep production online. If you're managing the morass of legacy software in production, this talk is for you.
Bio
Tyler Schade is a Distinguished Engineer at GEICO, where he leads software engineering and architecture for GEICO's infrastructure. An avid proponent of open source software, Tyler currently serves on the Istio steering committee and as a maintainer for a number of CNCF-hosted networking and security projects. Prior to GEICO, Tyler held roles at Solo.io, Ironnet, and Comcast, building deep networking, security and distributed systems experience. He lives in Philadelphia with his wife and their newborn.
Paul-Henri Pillet
Gatling
Don't RTFM, use AI instead for your first Gatling tests!
Abstract
AI has dramatically reduced the complexity of starting a new code project. Today, you don't read the docs first - you ask your coding agent. In this talk, we'll show how this applies to performance testing. How developers with zero Gatling experience can write realistic load tests in minutes, using AI as their entry point instead of documentation, and why this changes how teams adopt performance testing altogether.
Bio
Paul-Henri Pillet is the CEO of Gatling, one of France’s most successful open source technology companies and a leading provider of high performance load testing solutions. Since joining Gatling in 2014, he has helped grow the platform into a globally trusted tool used by hundreds of thousands of organizations to ensure the scalability and reliability of their applications. Based in Paris, Paul-Henri combines a background in business, public policy, and entrepreneurship, with degrees from HEC Paris and Freie Universität Berlin.
Michael Levan
Solo.io
Sandboxed Agents: Building Isolated, Governed and Scalable AI
Abstract
Agents need to be isolated for what they can and can't access (files/folders, production envs, LLMs, MCP tools, runtimes, etc. - think autonomous and standard Agents) and at the same time, scale up and down based on load, if/how the Agent is being used, and how it's being used.
By default, Agents have access to every file, LLM, command, and authN/Z implementation that you do. At the same time, you're running Agents in a scalable, performant, and reliable way with a proven out orchestration/clustering platform.
Example: You want to run 200+ Agents, but only spread across 6-8 k8s Pods. The Agents still perform as well as expected while keeping the underlying infrastructure costs (both resources and actual financial costs) low.
In this session, you will learn about the OSS project that makes this happen - Agent Substrate, with key topics including:
Build on top of k8s primitives (clustering of resources for performant sandboxed Agents).
Implement a specialized and efficient a Control/Management Plane that's built for the world of agentic.
How Substrate Actors running Agents are 90% more efficient (proven in benchmarks) for cost optimization in comparison to Agents in k8s Pods.
Efficient and much smaller footprints for running Agents via Actors.
Bio
Michael Levan translates technical complexity into practical value. He is a Principal AI Architect and Forward Deployed Engineer in the AI and Platform Engineering space who spends his time working with organizations around the globe on technical implementation and strategy. Michael is also a Microsoft MVP, 4x published author, podcast host, international public speaker, AAIF Ambassador, CNCF Ambassador, and was part of the Kubernetes v1.28 and v1.31 Release Team.
PJ Hagerty
WWT
Why Dev Tools Still Matter
Abstract
With more and more tools abstracting from a developer's workflows, understanding how git visualization tools help - not simply using it - is more important than ever. In this talk, we take a look at the history of git workflows, the basics of git, and the renaissance of understanding version control in a world gone mad for “vibe coding”.
Bio
PJ Hagerty is a well-known figure in the tech industry, particularly within the developer relations and DevOps communities. He is recognized for his work as a developer advocate, community builder, writer, and speaker. PJ founded DevRelate.io, a company focused on helping tech organizations build communities and enhance developer relations. He has a strong presence in the open-source and tech communities, often speaking at conferences and producing content on topics such as software development, DevOps, and community engagement.
Dor Amir
Nadir
Cache Is King: The Hidden Cost of LLM Optimization
Abstract
A cheaper model does not always mean a cheaper run. Routing between models, trimming context, splitting work across agents, or adding tools and skills mid-run can look like smart optimizations. But when those changes disrupt prompt-cache reuse, they can turn discounted, reusable context into input you pay to process again—undermining the savings you expected. OpenAI Developers
This talk explores why prompt caching deserves to be an architectural consideration, not an afterthought, in production LLM systems. We’ll examine how caching interacts with model routing, context management, subagents, and dynamic tool loading. Through concrete examples, we’ll compare preserving a warm cache with starting fresh, explore when breaking the cache is worth the trade-off, and show how to make these decisions based on the economics of an entire run rather than the price of an individual request.
Attendees will leave with practical patterns for cache-aware workflows and an observability checklist covering cache reuse, cache-read and write costs, latency, and cost per successfully completed task. The takeaway: don’t optimize away your biggest savings. Optimize the whole run—not just the token count.
Bio
Dor Amir is the founder of Nadir, an LLM infrastructure company focused on making AI systems more efficient. Previously, he worked on machine learning and personalization at Dropbox, Amazon, Guesty, and Fiverr. His work focuses on production AI systems, LLM routing, agents, and reducing the cost and latency of running AI at scale.
Romaric Philogene
Qovery
SRE in the Era of AI: Lessons Learned from Hundreds of Teams I've met Over 3 Years
Abstract
Over the past three years, I've collaborated with hundreds of SRE teams as they've integrated AI into their workflows. What has truly evolved - and what remains the same? This talk will provide practical insights and observations from the forefront of AI adoption in infrastructure. I'll discuss:
The evolving skill set: What SREs need to learn today and which skills stand the test of time
AI in incident management: Identifying where AI agents add value, where they fall short, and how to distinguish between the two
AI SRE: What's real, and what's just hype
Bio
Romaric Philogene is CEO & Co-founder of Qovery, the Agentic infrastructure platform powering hundreds of companies like Alan, Talkspace, Zoom..With 20+ years in infrastructure engineering, he started his career when cloud didn't exist - managing physical servers in HFT, getting 2 AM pages, and building runbooks from scratch. Prior to Qovery (founded 2020), he was CEO of two other companies self funded and sold to major brands in ERP for automotive.
Elliot Gardiner
Mirror Panel Inc.
Production Has Observability. Your Coding Agent Doesn’t
Abstract
Claude Code, Codex, and other coding agents are becoming long-running, tool-using systems that read repositories, execute commands, modify code, call external services, and make decisions with limited human supervision. Yet most teams observe them primarily through token counts and final pull requests. This talk applies familiar SRE practices to agentic development. We’ll instrument a coding-agent workflow with OpenTelemetry, examine traces of tool calls, failures, retries, tests and code changes, and look at what those traces reveal about reliability that the final output hides. We’ll cover practical failure patterns, useful signals, and where traditional observability concepts translate well - and where they don’t.
Bio
Elliot Gardiner is a software engineer and former principal SRE with 13 years of experience building and operating large-scale systems. He is the founder of Code Telemetry, an observability platform for AI coding agents, and co-founder of Mirror Panel, a synthetic research platform focused on measuring and improving the reliability of AI-generated audience insight. His current work sits at the intersection of AI systems, observability, developer infrastructure, and reliability.
Denys Zhak
Imubit
What AWS Lambda Was Hiding: Runtime Assumptions Exposed by a Migration
Abstract
Moving a Lambda into a service looks straightforward: wrap the handler in an API or a queue consumer and deploy. That’s what we did at first when we migrated seven Python data-ingestion Lambdas to k8s, and the early tests passed, including concurrent deliveries. As we moved more customers over and left the services running for days, code that had worked on Lambda for years started breaking. Liveness probes timed out, pods kept running out of memory during DLQ replay, tmp files collided, connections dropped and credentials expired.
I’ve organised the talk around four promises we relied on by using Lambda: blocking was free, every delivery had the place to itself, nothing lived long enough to go bad, and the outside world saw a different you. We relied both on Lambda’s documented behaviour and on what we’d observed while running our application there. Moving to services forced us to revisit how we managed concurrency, connections and credentials once deliveries shared resources and processes lived longer.
I’ll use evidence from the migration to explain each promise, including how fixing the blocking calls made the memory problem easier to trigger. I’ll also cover what to test alongside a rollout, including replay, sustained load and credential rotation. I’ll finish with a checklist for identifying what your application depends on its runtime to do, and what you’ll need to account for when that runtime changes.
Bio
Denys Zhak is a software engineer with nine years of experience in backend engineering and distributed systems, focused on production reliability. At Imubit, he builds cloud and on-prem software for industrial machine-learning deployments. He led the six-month migration of seven Python ingestion Lambdas into services described in this talk. Outside work, he contributes to open source. Much of that work has been on Astral’s Ruff and ty. He writes about production engineering at denyszhak.com.
Michael Posso
TheRobotAge
Your Code Deploys. Your Docs Drift: Can AI Agents Fix That?
Abstract
As engineering teams accelerate software delivery with AI, documentation struggles to keep pace. Outdated runbooks, API references, and technical guides can create operational risks, slow incident response, and undermine platform reliability.
What if documentation could evolve alongside the code itself?
This talk explores how AI agents can detect documentation drift, identify changes in code and infrastructure, and automatically propose updates as part of the engineering workflow. We'll examine integrations with tools like Jira, Confluence, and GitHub through Model Context Protocol (MCP), along with automated validation, CI/CD integration, human approval, and governance.
Through a practical example, attendees will learn how to approach documentation as a continuously maintained, verifiable component of software delivery rather than an afterthought.
Bio
Michael Posso is a technology leader, engineer, and educator with over 15 years of experience in web development, cloud platforms, and digital products. He currently serves as Vice President at JPMorgan Chase, focusing on product engineering, go-to-market (GTM) strategy, and AI platform adoption. Previously, he led web engineering at AECOM, developing enterprise web experiences for global audiences. He is also the founder of The Robot Age, where he explores robotics, human-robot interaction, and the future of intelligent systems.
Giovanni Martinez
Tiger Data
Agents Can Write Migrations. They Shouldn’t Ship Them.
Abstract
AI agents can write migrations, but production changes need guardrails. This live demonstration walks through an agent-assisted PostgreSQL workflow using Tiger Cloud and the Tiger CLI: provisioning an isolated development service, generating versioned migrations, and testing correctness and performance before human-approved production deployment. We’ll cover scoped access, automated checks, lock and rewrite detection, deployment monitoring, and rollback criteria. Attendees will leave with a practical blueprint for accelerating database delivery without giving agents unchecked authority over production.
Bio
Giovanni Martinez is a Database Support Engineer at Tiger Data and an AWS Community Builder with more than 10 years of PostgreSQL experience. He specializes in database performance, production reliability, cloud-native PostgreSQL, TimescaleDB, and agent-assisted database operations. Through The Postgres Guy and his open-source iqtoolkit project, Giovanni creates practical tools and educational content that help engineers troubleshoot, optimize, and operate PostgreSQL systems safely.
AJ Meyers
AJ Meyers
Always On SRE CoWorker
Abstract
Growing alerts, new sources, growing volumes of telemetry. This talk shows an SRE Coworker that is always-on to correlate and detect subtle anomalies across sources, investigate complex hypotheses, and surfaces root causes automatically. Attend this session to discover how your team can wake up to zero alerts and fully explained incidents while you sleep.
Bio
AJ Meyers is a technology strategist, architect, and leader who helps businesses turn advances in AI, data, and emerging technologies into opportunities for growth and competitive advantage. His career spans business strategy, product development, organizational leadership, and enterprise operations, combining technical depth with experience building teams and directing large-scale technology investments.
Currently a Principal Solutions Architect at Elastic, AJ brings extensive experience across leading technology companies and consultancies, including Teradata, FICO, Deloitte Digital, and EY.
Marcelo Lorenzetti
Guild Systems
Who Stops the AI Agent at 2 AM?
Abstract
As AI agents move from experimentation into production, they are increasingly capable of executing commands, accessing enterprise systems, modifying infrastructure, and making operational decisions without continuous human supervision.
But what happens when an autonomous agent makes the wrong decision at 2 AM? Who detects it, who has authority to intervene, and how do we stop the action before it becomes a production incident?
Traditional SRE practices provide powerful mechanisms for observability, reliability, incident response, and recovery. Autonomous AI introduces additional challenges involving agent identity, dynamic authorization, decision accountability, and runtime enforcement.
This session introduces a practical architecture for bringing AI governance directly into production operations. We'll explore how OpenTelemetry, agent registries, identity-aware policies, authorization gateways, human approval workflows, runtime monitoring, and automated kill switches can work together to make autonomous systems safer and more reliable.
Through a realistic production incident scenario, we'll examine how an AI agent's proposed action can be evaluated, authorized, blocked, escalated, and documented before unintended consequences occur.
Attendees will leave with actionable architectural patterns for connecting SRE practices with enforceable AI governance.
The central message: Observing an AI agent is not the same as controlling it. Reliable autonomous systems require both.
Bio
Marcelo Lorenzetti is Chief AI Officer and Principal AI Architect at Guild Systems, specializing in enterprise AI architecture, agentic AI governance, cloud infrastructure, and operational resilience. With extensive experience leading enterprise technology architecture and AI transformation initiatives in financial services and other regulated environments, he focuses on turning AI governance principles into enforceable production controls. His work spans autonomous agent orchestration, identity and authorization, observability, runtime policy enforcement, operational risk, and AI economics. Marcelo also serves on the New York Law School AI Blue Ribbon Committee and is an author and speaker on enterprise AI, responsible innovation, and technology governance. His approach bridges strategic governance and practical engineering, helping organizations deploy autonomous AI systems that are reliable, accountable, and operationally controllable.
Raymond Carlson
Carlson BCM
Observing the Enterprise, Not Just the Stack
Abstract
You can observe every component of an enterprise and still not understand the health of the enterprise. SRE practice tells us to set objectives around what users care about, but those objectives rarely connect to a governed account of what the enterprise actually does. The missing object isn’t another metric, dashboard or SLO. It’s the business capability: the enduring definition of what the enterprise must be able to do, independent of the systems that deliver it. This talk sets out how a capability-based topology, running from business capability through applications and data to infrastructure, can become the organizing foundation for service health, semantic analysis and AI-driven diagnostics. It compares three approaches: adding capabilities to existing observability tools, making the capability model the organizing structure for service health, and using AI agents to engineer observability against topologies that change constantly. The argument throughout is that capabilities are the layer worth designing for, because they remain stable as the technology beneath them changes.
Bio
Raymond Carlson is the founder and principal of Carlson BCM, a business capability architecture practice for insurance and financial services. Over three decades he has led design authority on enterprise transformations across insurance, banking and capital markets, connecting strategy, operations and technology to a single governed model of what the enterprise must be able to do. His recent work applies that model to observability and service health, anchoring enterprise topology, operational data and AI-driven diagnostics to the business capabilities they support. He is the author of the Carlson BCM framework and publishes the weekly newsletter Carlson BCM Insights.
Roy Resh
Nibbana
Green Dashboards, Wrong Cows: Silent Failures in an AI-Built Data Platform
Abstract
Our most critical upstream is a decades-old herd-management program on a farm-office PC. One day our dashboard showed a fraction of a farm's real milking herd. Every service was healthy, every job exited 0, and no alert fired. We heard about it from the farm, not from our monitoring. Those numbers feed emissions reporting, so a wrong count is a wrong carbon claim. We built the platform largely with AI coding agents. I will walk through incidents where every component was correct and the system was still wrong: a history purge that looked like cows leaving, one agent uploading several farms' data, a concurrency guard that swallowed updates, and "idempotent" migrations that never converged. Finally, I will show the AI agent we're layering on our alerting jobs to flag these failures as they happen and learn from what it misses, and close with a checklist for finding silent failures in your own pipelines.
Bio
Roy Resh is a Senior Software Architect at Nibbana, where a small team uses AI coding agents to build a data platform for dairy emissions reduction. Roy led the incident investigations described in this talk. Before Nibbana, Roy spent over a decade at Trax Retail, most recently as VP of Engineering leading 80+ engineers through a monolith-to-microservices migration. Roy holds an MSc in Physics from the Hebrew University of Jerusalem.
Yossi Eliaz
Incredibuild
Reliability Lessons from Building Sandboxes for AI Agents
Abstract
AI agents now run thousands of ephemeral coding tasks a day, and each one needs a clean, fast, trustworthy environment to execute in. This talk shares hands-on lessons from building secure execution infrastructure for AI workloads at Incredibuild and islo.dev, covering CI runners, sandboxes, snapshots, caching, traces, and reproducible environments. It walks through a real sandbox provider for ephemeral task isolation, presented at an Apache Airflow Monthly Town Hall, and a meta-harness loop that took a coding agent from 0 of 5 to 5 of 5 passing tasks in about two seconds across four proposer steps. Drawing on earlier container security work at Twistlock, the talk covers what breaks when isolation, caching, and observability have to work together at CI speed.
Bio
Yossi Eliaz is a Principal Engineer, AI Systems at Incredibuild, building secure execution infrastructure for AI workloads: CI runners, sandboxes, snapshots, caching, and reproducible environments. He previously worked on container security at Twistlock. He holds a PhD in Physics and an MSc in Computer Science.
Tom Elliott
Modal
Degraded is better than down
Abstract
Everything breaks. Bad deploys, traffic spikes, cloud outages, eventually some component will go down. When architecting a system, our instinct can be to try and eliminate any possible impact. But this can add complexity that paradoxically makes failures more likely, or make recovery harder. Let's look at an alternative: embracing degradation to cushion against a total outage. We'll look at a real world example of multi-region routing, explore a catalogue of failure modes and trade-offs that can blunt them, and highlight when compromise isn't an option. Fail safe without having to be failure-proof!
Bio
Tom Elliott works on the Platforms team at Modal, with responsibility for shared infrastructure, observability, CI/CD and developer productivity tooling. Over the course of 20 years in the industry, Tom has worked within organizations of various sizes and levels of operational maturity. When not building infra for infra, Tom occasionally writes about the field at telliott.me.
Lesley Cordero
The New York Times
Actionable Observability
Abstract
While observability is the first step towards building observable systems, monitoring is what enables us to action on the telemetry collected in a more automated way. Previous approaches to monitoring have relied primarily on infrastructure and service metrics, but modern approaches have embraced the idea of monitoring based on metrics that reflect the user experiences that consider business impact more accurately. In this, we're enabled to drive clarity over reliability targets, effectively prioritize reliability improvements, and hold ourselves accountability to customers.
This talk will elaborate on those differences and review traditional and new approaches to monitoring, including the role of Service Level Objectives (SLOs). Additionally, we’ll cover the following:
Observability vs Monitoring
Metrics and the types of Metrics & Monitors
The benefits of an SLO-based approach to monitoring
Effective Accountability with Error Budget Policies
Bio
Lesley Cordero is currently a Staff Software Engineer, Tech Lead at The New York Times. She has spent the majority of her career on edtech teams as an engineer, including Google for Education and other edtech startups. In her current role, she is focused on resilience engineering, observability, shared platforms, and building excellent teams by setting reliability vision & strategy across The Times, improving our observability footprint, and cultivating culture that builds with the most vulnerable employees in mind first. She shows care for others by holding them accountable to the best versions of themselves – and by buying them the occasional bubble tea.
Itamar Knafo
Dalton AI
You Build It, You Run It Is Dead: Reliability When the Author Is an Agent
Abstract
You build it, you run it. Fifteen years of SRE gospel, and almost nobody questions it. The rule was never about writing code. It was about a loop. Whoever wrote it understands it, so whoever wrote it carries the pager. Understanding was the point. The pager was just enforcement. Agents cut that loop. The code still ships. The author is gone. I take the assumption apart, one joint at a time. No human author, so who holds the pager. Nobody who can explain it, so the author cannot answer the 3am question. Review cannot keep up with the output, so your gate is a rubber stamp. Then the fixes that do not work, and why each one folds. Human in the loop. Make the agent better. Keep agent code out of prod. Then the reframe. If writing it no longer means understanding it, reliability has to come from somewhere else. Operability stops being something you hope the author cared about, and becomes something you check for. One real incident carries the argument. A single S3 copy call, a 5 GiB limit nobody checked, a migration that never sealed, and a cluster crash-looping while the log line said migrating objects: <nil>. None of it was hard. Nobody could see it. The principle is not dead. The assumption under it is.
Bio
Itamar Knafo is the co-founder and CEO of Dalton AI. Previously, has spent five years building and leading software, DevOps, observability, and operations teams. He is a hands-on engineer with a deep background of cloud infrastructure, and open source, and now focuses on applying AI to production engineering.
William McLean
Glitnir Ticketing
Inference on Cloud TPU
Abstract
New to TPUs? This talk starts with what a Cloud TPU is, how it differs from a GPU, and how to get one on Google Cloud. Then it moves to a live demo: an MCP server that lets an AI agent find TPU capacity, create a TPU VM, and start vLLM on it. We send the server a few requests, watch it answer, and tear everything down so nothing keeps billing. You leave with the steps to run your first model on a TPU.
Bio
CTO @ Glitnir Ticketing | GDE AI+Cloud ☁️ | AWS Community Builder AI | Google Cloud Ambassador | 9x Google Cloud Certified
Anand Mattah Chenna Kesavalu
Grainger
Navigate RUM Without Limits: Cutting 80% of Browser Telemetry Noise While Improving Observability
Abstract
Modern observability platforms make it easy to collect browser telemetry. The real challenge is deciding what not to collect.
At Grainger, our digital platforms process approximately 1.5 billion requests per day across multiple brands, countries, and customer-facing applications. As our Real User Monitoring (RUM) adoption expanded, we discovered that a significant portion of our telemetry was generated by bots, synthetic monitoring solutions, automated testing platforms, internal tooling, and duplicate browser events. While these events increased observability costs, they provided little value when understanding real customer experiences.
This session shares our journey modernizing browser observability during a Datadog RUM migration and transforming RUM from a "collect everything" model into a scalable, business-focused observability platform.
Attendees will learn how we:
• Identified and filtered bot, synthetic, and automated traffic before it reached our observability platform
• Reduced browser event ingestion by nearly 80% (7.81 million to 1.67 million events) while maintaining visibility into real user experiences
• Improved session accuracy by addressing split-session behavior, cookie inconsistencies, and telemetry fragmentation
• Implemented intelligent filtering across edge, application, and browser layers to reduce noise and improve signal quality
• Balanced engineering observability, business analytics, compliance requirements, and platform costs
• Established a repeatable framework for determining which browser telemetry delivers operational value and which should be excluded
Beyond the technical implementation, this talk explores the broader challenge facing modern SRE and observability teams: collecting the right data rather than simply collecting more data.
The result was a cleaner signal, faster incident investigations, reduced observability spend, and greater confidence that our dashboards, alerts, and analytics reflect real customer behavior—not automated traffic.
Whether you use Datadog, OpenTelemetry, New Relic, Dynatrace, Elastic, or another observability platform, you'll leave with practical techniques and architectural patterns to navigate RUM without limits.
Bio
Anand is a Staff Software Engineer and Platform SRE at Grainger, where he serves as the enterprise subject matter expert for CDN, edge security, bot management, observability, and web performance. He is responsible for reliability and performance initiatives supporting more than 1,000 services and APIs across Grainger's digital platforms.
Anand has over 20 years of experience in software engineering, site reliability engineering, and cloud infrastructure. He has led large-scale implementations involving Datadog, Akamai, DataDome, AWS, and enterprise observability platforms. His work includes browser observability optimization, bot mitigation, edge security modernization, and platform reliability engineering. Anand is also the author of a several guest blogs at leading vendors including Datadog RUM [https://www.datadoghq.com/blog/grainger-rum-costs-from-bot-traffic/]
He holds numerous industry certifications including AWS Machine Learning Speciality, AWS Solutions Architect Professional, AWS Certified Generative AI Developer - Professional, Datadog Certified Professional, Github Copilot, Terraform Associate, Splunk certifications, Akamai Professional, Gremlin Certified Chaos Engineer and Google GenAI credentials etc.,
A frequent conference speaker, Anand has presented at multiple Grainger technology conferences on observability, bot management, and security engineering.
Manan Chawda & Nikunj Doshi
Experian
Paying for the Worst Day: Sizing Standby from RTO
Abstract
We keep spare infrastructure running for the worst day of the year. It consumes resources every other day. We wanted to know how much standby capacity we actually need to buy the recovery objective we promise. We built the same Kubernetes service with three disaster recovery strategies: pilot light, warm standby, and hot standby. Then we failed the primary region and measured what happened. The tradeoff was visible on both sides. Keeping more capacity ready reduced recovery time, but increased energy use while waiting for an incident. Keeping less capacity idle saved energy, but made recovery depend on provisioning, autoscaling, cache warming, and downstream capacity exactly when the system was already under stress. We'll compare each strategy using recovery time, p99 latency, failed requests, error budget burn, and workload energy measured during both idle and recovery periods. We will also look at where scaling from a smaller standby stops being fast enough to satisfy the RTO. We'll finish with a practical method for sizing standby capacity from an RTO instead of assuming that more redundancy is always better.
Bio
Manan Chawda is a software engineer at Experian focused on distributed systems, cloud-native platforms, and production reliability. He has built backend and data infrastructure across AdTech and FinTech and holds an M.S. in Computer Science from New York University. Nikunj Doshi makes Solutions simpler, secured, reliable and accessible. Nikunj Doshi is an accomplished IT Architect with over 10+ years of professional experience and more than a decade of deep specialization in Open Source technologies, Cloud Architecture, Artificial Intelligence, and Cybersecurity. His career reflects a rare combination of technical excellence, research contribution, global project leadership, and sustained community impact.
