Running ML workloads on Kubernetes can be a burden to manage, especially when the usages tend to diversify. Batch inference will involve different constraints as experimentation, evaluation or serving ML models for user-facing applications.
To answer those questions, we will see in this talk how combine the standard orchestration approach of Argo Workflows with the autoscaling super powers of Karpenter, a tool developed by AWS and given back to the community. We'll dig into the difficulties of scaling GPUs and how Karpenter solves them, with a return of experience of Pruna's computing platform.
MLOps Engineer working at Pruna AI, I am interested in efficient, reliable & robustly beneficial AI. My work is centered around building ML development platforms, and provide relevant tooling to build & deploy robust ML models in applications.
