This session delivers critical insights into leveraging Kubernetes Operators in the data space. I'll cover the Kubernetes operator pattern, KubeBuilder, data governance, how we built our connector ecosystem, and many more.

Building a data streaming platform that can scale with your organization while maintaining developer productivity is challenging. In this talk, I'll unpack the architectural decisions and engineering challenges of building a cloud-native streaming platform from scratch using Kubernetes Operators. Through this journey - from initial design to production scale, we'll explore transforming the operator pattern into a foundational piece of our data infrastructure, powering thousands of streaming workflows daily. Beyond the technical implementation, I'll share insights about building developer experiences that scale. We will cover the architectural patterns used and the extensible connector ecosystem that enabled our platform to handle millions of daily events without changing its core design.

Elad Leev is a Staff Engineer and open-source enthusiast with over 10 years of experience managing complex production systems. He has expertise in distributed systems, databases, and data streaming technologies. Elad is using Kafka in production since 2016, and he is a strong advocate for data streaming solutions, particularly with technologies like Kafka, Flink, Debezium and Kafka Connect.