In March 2025, Cloudflare's R2 went down for an hour. Production was using an old secret version. The backend had rotated to the new one. Secret-config drift. One hour. Global outage. Because a secret and its config lived in different places, managed by different tools, versioned separately. We stopped treating them as special. Secrets are just config that happens to be encrypted. Same git repo. Same commit. Same deploy. Same rollback. Encrypted with sops-nix, baked into a NixOS image at build time — full OS, not just the app. Deployed via Incus, no SSH. Key pushed separately. Decrypted locally, delivered through systemd credentials. No separate secret server. No separate pipeline. No version mismatch. Rotation doesn't disappear — it gets better. A secret rotation is a git commit, a build, a deploy. You know exactly when it happened, who did it, and which systems have it. The audit trail is git history. No logs to tamper with, no API calls to trace. 30 minutes. How it works, where it hurts, and why this model gets stronger as your system gets more complex.
I run Plumelo, a consultancy working with NixOS, Terraform, and Incus in production. Based in Romania. Big believer in self-hosted and open source. Enjoy playing around with hardware, routers, and networking.
