Observability data isn't homogeneous. Security logs require needle-in-haystack searches with multi-year compliance retention. Kernel logs are uncompressible text. Structured logs enable fast aggregations, while semi-structured logs explode cardinality. Traces demand different access patterns entirely.
Modern requirements compound this. Observability must join with other data sources. Agentic AI systems generate massive volumes of unstructured and semi-structured logs and traces. Big data platforms have emerged as popular storage alternatives.
Forcing everything into one system creates impossible tradeoffs: slow queries, runaway costs, frustrated users.
At Airbnb and Slack, operating thousands of tenants across hundreds of clusters, we built a polystore architecture routing workloads to specialized engines, unified behind a single query interface. This required changes across the entire stack: instrumentation, collection, storage, and query layers.
This talk shares routing criteria, backend tradeoffs, and techniques for unified querying. Attendees will learn to optimize observability for better performance and lower costs.
Suman Karumuri is a systems and infrastructure engineer focused on observability and distributed systems. He is the founder of KalDB, a serverless observability platform for agentic workloads. Previously, he was a Principal Engineer at Airbnb, where he led observability and reliability efforts, and held engineering roles at Slack, Pinterest, and Twitter, building large-scale logging, metrics, and tracing systems.
