RLHF rewards confident, complete answers. Production incident investigation requires the opposite: withholding judgment, maintaining competing hypotheses, and seeking disconfirming evidence. The training incentive points directly away from the task. At Cleric, we built an AI agent that investigates production incidents. When a cascading failure produces dozens of correlated anomalies, the agent latches onto the loudest signal and builds a coherent narrative around it. The reasoning is sound. The answer is wrong. We taxonomized our agent's production failures and found that premature convergence (the agent finding a plausible answer and stopping) is one of the most common failure modes. Red herrings in production aren't random noise. They're symptoms that look like causes: a downstream service timing out because an upstream database is slow. The timeout is the loudest signal. The database is the root cause. This talk covers the architectural patterns we use to counteract it: forcing multiple competing hypotheses before commitment, requiring evidence that distinguishes between hypotheses rather than confirms the leading one, and a devil's advocate layer that argues against the top hypothesis. We also cover causal inference techniques that make the problem tractable, cascading failures propagate with measurable latency through dependency graphs, and dozens of anomalies collapse into a short causal chain when you follow timestamps instead of severity.
Willem Pienaar is the Co-Founder and CTO of Cleric, where he builds autonomous AI agents that operate inside production infrastructure. He created Feast, the widely adopted open source feature store powering ML systems at companies including Cloudflare, Discord, Robinhood, Salesforce, and Shopify. Before founding Cleric, Willem led open source engineering at Tecton and built the ML platform team at Gojek, one of Southeast Asia's largest ride-hailing companies. He holds an MS in Computer Science from Georgia Tech.
