As software shifts from deterministic code to non-deterministic LLMs and autonomous agents, traditional CI/CD practices fall short. Treating probabilistic agentic workflows like rigid software forces tough choices around test coverage, pass rates, and build times. This talk explores how to modernize DevOps pipelines for the AI era. We'll examine how balancing mode l selection, evaluation costs, and pipeline speed affects both developer velocity and system reliability. We will dive into improvement strategies like intelligent task selection, lightwei ght judging, test tiering and eval prediction can reduce evaluation time and costs.
Hubert Chen is a San Jose–based Infrastructure Engineering leader and architect with over 30 years of experience scaling backend systems, SRE, and DevOps operations. Currently a Member of Technical Staff at Fidian focused on infrastructure for AI testing and cloud-managed Kubernetes, he brings extensive senior leadership from Palo Alto Networks as Distinguished SRE and Director of Infrastructure, where he managed hundreds of Kubernetes clusters across AWS and GCP, multi-petabyte data lakes, and tens of millions in cloud spend. His background also includes leading infrastructure and systems scaling at Branch and Shutterfly, supported by a BS in Computer Science from the University of Michigan.
