k0s - Kubernetes Without the Package Hell

kubeadm is a fucking nightmare. Install kubelet, then kubectl, then kubeadm, then containerd, then debug why they're all fighting over different socket paths. Spend an afternoon figuring out why the API server crashes on boot because some systemd unit didn't start in the right order.

k0s throws all that in one 165MB binary. Same Kubernetes APIs, just packaged by people who apparently understand how software distribution works. Download it, run it, cluster exists. No version compatibility matrix or hunting down the right containerd release.

Been testing it for a few months now on staging environments. The current release (v1.30.x series) is pretty solid - way less painful than our old kubeadm setup. Mirantis maintains it so it's not some random side project that'll disappear when the maintainer gets bored.

Everything crammed into 165MB

It's bigger than k3s but includes actual Kubernetes components instead of weird Rancher replacements. kube-apiserver, etcd, kube-scheduler, controller-manager, kubelet, containerd, CoreDNS - all the stuff that usually breaks when you mix versions.

containerd is way better than Docker's runtime. Docker would shit itself on networking edge cases or when you tried scaling past like 8 nodes. containerd just works without randomly dying when you restart pods.

Zero dependencies (mostly)

Works on pretty much any Linux with kernel 3.10+. That covers everything newer than CentOS 7, which is already ancient. Tested it on:

x86-64, ARM64: Runs fine. ARM64 Pi 4 handles small workloads well.
ARMv7: Pi 3 works but gets slow with more than a few pods.

Ubuntu/Debian: Just works.
CentOS/RHEL: Need to fuck with firewalld rules but runs fine after that.
SUSE: Haven't broken it yet.

The main gotcha is enterprise Linux with custom security shit. Spent 3 hours debugging one company's "hardened" RHEL where SELinux blocked containerd from accessing its socket. Had to figure out the right security context rules. Ubuntu just works without the security theater.

The quick way (for testing)

curl -sSf https://get.k0s.sh | sudo sh
sudo k0s install controller --single
sudo k0s start
sudo k0s kubectl get nodes

Works on Ubuntu 20.04+, CentOS 8+, most normal Linux. If you're behind a corporate firewall, the curl might fail - just download the binary from GitHub instead.

CentOS 7 needs yum install -y conntrack first or k0s bitches about missing kernel modules.

Don't believe the 1GB RAM minimum in the docs. That's for "hello world" demos. You need 2GB+ for anything real. Tried running it on a 1GB DigitalOcean droplet - started fine, died the second I deployed Prometheus. Also make sure /tmp has space for the binary extraction, learned that one the hard way.

Multi-node with k0sctl

k0sctl is like Terraform but for k0s clusters:

apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
spec:
  hosts:
  - ssh:
      address: controller-01.example.com
      user: root
    role: controller
  - ssh:  
      address: worker-01.example.com
      user: root
    role: worker
  k0s:
    version: "1.30.4+k0s.0"

k0sctl does the SSH bullshit for you - installs binaries, handles rolling upgrades, backs up etcd. Single YAML file manages the whole cluster.

The pain point: your SSH setup has to be perfect. k0sctl will hang for 10 minutes if passwordless SSH doesn't work, then give you "connection failed" errors that tell you nothing. Test ssh root@node1 'echo test' on every node first or you'll waste half your day debugging.

Also runs everything serially by default, so deploying to 20 nodes takes forever unless you bump the parallelism.

Other deployment options

Airgapped/corporate networks: Download the binary and container images separately. No surprise downloads during deployment that make security teams panic.

Docker containers: For dev work, run k0s in containers. Pretty clean way to test without fucking up your laptop.

Raspberry Pi: Works fine on Pi 4 with 4GB+ RAM. ARM64 support is solid. Pi 3 works but gets slow.

What you actually need

From running it on various VMs:

• Minimum for real work: 2GB RAM, 2 vCPU, SSD storage
• Memory usage: ~500MB idle, 1GB+ with actual workloads
• Startup: 20-30 seconds, longer with lots of pods

Tested on $10 Hetzner VMs, AWS t3.medium, old Dell servers. Works everywhere but don't cheap out on memory. 1GB VMs work for hello-world demos, start swapping with real apps.

Big gotcha: k0s extracts the entire binary to /tmp during upgrades. Had one fail because /tmp filled up and it took me way too long to figure out what happened.

Check the docs for more deployment details.

CNCF and corporate backing

k0s joined the CNCF Sandbox this year, which basically means it's not going to disappear overnight. Mirantis maintains it and sells enterprise support.

If you need someone to blame when things break, Mirantis offers support contracts with SLAs. They also do FIPS builds for government compliance requirements.

Having actual corporate backing is nice - means the project won't get abandoned when the maintainer gets bored or changes jobs.

Related tools

k0smotron: Manages multiple k0s clusters using Cluster API. Useful for fleet management if you're running k0s on edge devices.

MKE 4: Mirantis Kubernetes Engine is their enterprise k0s distribution. Same thing but with more enterprise features and a higher price tag.

Where k0s works well

Edge devices: Single binary is perfect when you can't install packages normally. Works fine on industrial gateways and ARM boxes.

Dev environments: Faster setup than kubeadm, easier cleanup than k3s. Good for testing.

Small companies: Want Kubernetes without a platform team. The docs are actually readable.

Hybrid deployments: Same binary works on AWS, VMware, bare metal. No cloud-specific bullshit to debug.

Community stuff

Community is decent - not huge like k3s but people actually help on Kubernetes Slack. Monthly office hours that are actually useful, not just marketing presentations.

Standard Kubernetes tooling works

Since it's just regular Kubernetes, all the normal tools work:

GitOps: Flux and ArgoCD install fine
Ingress: NGINX, Traefik work normally
Load balancing: MetalLB for bare metal
Storage: Standard CSI drivers - Longhorn, OpenEBS, Rook Ceph
Monitoring: Prometheus, Grafana, whatever you normally use
Service mesh: Istio and Linkerd work without special config

Bottom Line

k0s is Kubernetes packaged like normal software instead of 20 different packages that fight each other. Single binary, decent docs, corporate backing so it won't disappear.

Not perfect but removes most of the bullshit that makes Kubernetes deployment painful. If kubeadm upgrades have burned you before or you're tired of k3s making weird decisions, k0s is worth testing.