Moving a Lambda into a service looks straightforward: wrap the handler in an API or a queue consumer and deploy. That’s what we did at first when we migrated seven Python data-ingestion Lambdas to k8s, and the early tests passed, including concurrent deliveries. As we moved more customers over and left the services running for days, code that had worked on Lambda for years started breaking. Liveness probes timed out, pods kept running out of memory during DLQ replay, tmp files collided, connections dropped and credentials expired. I’ve organised the talk around four promises we relied on by using Lambda: blocking was free, every delivery had the place to itself, nothing lived long enough to go bad, and the outside world saw a different you. We relied both on Lambda’s documented behaviour and on what we’d observed while running our application there. Moving to services forced us to revisit how we managed concurrency, connections and credentials once deliveries shared resources and processes lived longer. I’ll use evidence from the migration to explain each promise, including how fixing the blocking calls made the memory problem easier to trigger. I’ll also cover what to test alongside a rollout, including replay, sustained load and credential rotation. I’ll finish with a checklist for identifying what your application depends on its runtime to do, and what you’ll need to account for when that runtime changes.
Denys Zhak is a software engineer with nine years of experience in backend engineering and distributed systems, focused on production reliability. At Imubit, he builds cloud and on-prem software for industrial machine-learning deployments. He led the six-month migration of seven Python ingestion Lambdas into services described in this talk. Outside work, he contributes to open source. Much of that work has been on Astral’s Ruff and ty. He writes about production engineering at denyszhak.com.
