Modern enterprise data infrastructure creates significant operational overhead for SRE teams, with organizations spending the majority of their engineering cycles managing ETL pipelines, data replication, and maintaining multiple storage systems across data warehouses, lakes, and specialized databases. This operational complexity directly impacts service reliability, increases mean time to recovery (MTTR), and creates numerous points of failure that challenge SLO achievement. This presentation demonstrates how data lakehouse architecture transforms operational paradigms by consolidating disparate data systems into unified, cloud-native platforms that dramatically reduce infrastructure complexity. Through practical implementation strategies, we explore how SRE teams can eliminate redundant data copies, reduce operational toil through automated governance frameworks, and achieve better observability through centralized metadata management. Key operational benefits include substantial reduction in data pipeline maintenance overhead, simplified monitoring through unified observability stacks, and improved incident response through consolidated failure domains. The architecture leverages open table formats like Delta Lake and Apache Iceberg to ensure vendor neutrality while enabling automated backup, recovery, and compliance workflows. We examine five critical architectural layers from an operational perspective: scalable cloud storage foundations with built-in redundancy, self-healing metadata catalogs that reduce manual intervention, automated semantic layers that abstract operational complexity, and optimized query engines that provide consistent performance under varying loads. Attendees will learn practical migration strategies for reducing operational complexity, frameworks for measuring reliability improvements through SLI/SLO metrics, and automation patterns for minimizing manual operational overhead. This session provides actionable insights for SRE teams seeking to modernize data infrastructure while improving service reliability and reducing operational burden.

Piyush Dubey is a technology professional with over a decade of experience architecting and developing large-scale, distributed systems in cloud-native environments. Currently serving as a Senior Software Engineer at Microsoft since March 2021, he specializes in building high-performance data platforms, scalable microservices, and lakehouse solutions using technologies including Apache Spark, Delta Lake, Apache Iceberg, and Hive. At Microsoft, Piyush leads the Metadata Interoperability initiative to support open table formats in OneLake, enabling seamless integration with external data platforms like Snowflake. He designed and developed the Table Preview API to enable data governance and sensitivity labeling across OneLake datasets, supporting GDPR and CCPA compliance. As part of the founding team for OneSecurity, he built the OneLake Data Access Roles framework to enable fine-grained access on data artifacts. Prior to Microsoft, Piyush worked at Amazon on the Display Advertising team developing APIs for ad delivery and publishing across Kindle devices, and contributed to data compliance APIs supporting GDPR and CCPA requirements. At ServiceNow, he designed and implemented a scalable onboarding automation workflow engine that became a full-fledged product marketed as Employee Workflows, and led the design of a web-based PDF editor that digitized signature workflows. During his time at Pearson, he led the redesign and migration of a legacy SOAP-based address verification service to a RESTful API, resulting in a 65% performance improvement. Piyush holds a Master's degree in Computer Science from the University of Iowa and a Bachelor's degree in Information Technology from Rajiv Gandhi Technical University. He holds a patent for Sound Assessment and Remediation and has expertise across multiple programming languages and cloud platforms including Azure, AWS, and Google Cloud. He is based in Boston, MA.