Modern AI agents don’t fail like traditional services, they fail across chains of tool calls, retrieval steps, and model decisions. There's no shortage of AI agents to pick from, but once you deploy an agent, the next hard problem is operability: What happened? What took so long? Which tool caused an error? Did the agent exfiltrate data or misuse permissions? In this session, you’ll see a practical observability architecture for AI agents using OpenTelemetry and MLflow. From how to correlate a user request with every tool invocation, to enforcing consistent trace context across agent to orchestrator to tool servers, and deriving agent-specific signals (overall success rate, step latency, retries, “time to first action,” and token/tool cost). We’ll run a live demo of a real end-to-end flow with production checklists for dashboards, alerts, redactions, and AgentOps guardrails that make agents trustworthy.
Cedric Clyburn (@cedricclyburn), Senior Developer Advocate at Red Hat, is an enthusiastic software developer with a background in Kubernetes, DevOps, and container tools. Focused on open-source software, he both contributes (e.g., Podman, vLLM) and enjoys speaking, with prior experience at Devoxx, WeAreDevelopers, The Linux Foundation, and more. Cedric also spends (too much) time creating video and written content helping developers learn new topics in emerging technologies, with over 2M+ views online. He’s based in New York City and is an organizer of the local Kubernetes Community Day.
