There’s a tension between developers/engineers (who want to send ALL the data to their tools of choice) and the monitoring/o11y teams (who must carefully manage the egress, ingest, and storage costs of those same tools). This creates issues with o11y data volume and data / instrumentation duplication, leaving everyone in a quagmire of a quandary: How to get the right data into the proper tool(s), without blowing up the budget? Even in an OSS context where every tool is “free”, there are still costs to be considered and managed. There is also the risk that the desire to keep costs low will lead to dropping data that doesn’t appear necessary, until the exact moment when it’s needed, at which point it’s suddenly invaluable. In this talk, I will first lay bare - with specific examples - the uncomfortable truth that in the world of o11y solutions there’s no one-size-fits-most, let alone one-size-fits-all. Even with the advances in OTel maturity, robust observability means having more than one (or even 3) tools doing the job. Then I’ll explain how telemetry pipelines not only solve the basic problems, they create a more robust and flexible observability environment overall. I’ll show you - once again with specific examples - ways various telemetry types can be distributed among multiple tools in ways that allow you to retain control over both the data streams and your budget. I will also give you strategies to manage (and even reduce) the administrative overhead that could arise once your organization sees how effective pipelines are, and everyone wants their own.
A full speaker profile is available on adatosystems.com In my sordid career, I have been an actor, bug exterminator and wild-animal remover (nothing crazy like pumas or wildebeests. Just skunks, snakes, and raccoons.), electrician, carpenter, stage-combat instructor, ASL interpreter, and Sunday school teacher. Oh, yeah, I've also worked with computers. While my first keyboard was an IBM selectric, and my first digital experience was on an Atari 400, my professional work in tech started in 1989 (when you got Windows 286 for free on twelve 5¼” when you bought Excel 1.0). Since then I've worked as a classroom instructor, courseware designer, helpdesk operator, desktop support staff, sysadmin, network engineer, and software distribution technician. Then, about 25 years ago, I got involved with monitoring. I've worked with a wide range of tools: Tivoli, BMC, OpenView, janky perl scripts, Nagios, SolarWinds, DOS batch files, Zabbix, Grafana, New Relic, and other assorted nightmare fuel. I've designed solutions for companies that were modest (~10 systems), significant (5,000 systems), and ludicrous (250,000 systems). In that time, I've learned a lot about monitoring and observability in all it's many and splendid forms.
