Operating System CentOS

Deploy Kubernetes with kubeadm on AlmaLinux 9 and Rocky Linux 9

Build a Kubernetes cluster with kubeadm on AlmaLinux 9, Rocky Linux 9 or CentOS Stream 9 using containerd, the pkgs.k8s.io repository and Flannel.

This guide builds a small Kubernetes cluster with kubeadm: one control-plane node and one or more worker nodes running AlmaLinux 9, Rocky Linux 9 or CentOS Stream 9, with containerd as the container runtime and Flannel as the pod network. At the end, kubectl get nodes shows every node as Ready and the cluster can run workloads.

CentOS 7 reached end of life in June 2024 and its old Kubernetes package repositories are gone. Kubernetes packages now come from the community repository at pkgs.k8s.io, and Docker Engine is no longer needed as the runtime: the kubelet talks to containerd directly. The steps below follow the current kubeadm installation documentation.

Requirements

  • Two or more servers running AlmaLinux 9, Rocky Linux 9 or CentOS Stream 9, each with at least 2 GB of RAM. The control-plane node needs at least 2 CPUs.
  • Full network connectivity between all nodes, and a unique hostname, MAC address and product_uuid on each node.
  • A user with sudo rights on every node.

The examples use 203.0.113.10 for the control-plane node (cp1) and 203.0.113.11 for a worker (worker1). Unless a section says otherwise, run each step on every node.

Prepare every node

Set the hostname

Give each node its own name, for example on the control-plane node:

sudo hostnamectl set-hostname cp1

Turn off swap

By default the kubelet refuses to start when swap is active. Disable it now and comment out any swap line in /etc/fstab so it stays off after a reboot:

sudo swapoff -a
sudo sed -i '/\sswap\s/ s/^/#/' /etc/fstab

Set SELinux to permissive

The kubeadm documentation sets SELinux to permissive mode so containers can access the host filesystem, which some network plugins require:

sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

Enable packet forwarding and load br_netfilter

Kubernetes needs IPv4 forwarding, and Flannel needs the br_netfilter kernel module:

echo br_netfilter | sudo tee /etc/modules-load.d/k8s.conf
sudo modprobe br_netfilter

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
EOF

sudo sysctl --system

Confirm the setting:

sysctl net.ipv4.ip_forward

Open the firewall ports

On the control-plane node, open the API server, etcd, kubelet, controller-manager and scheduler ports, plus Flannel's VXLAN port:

sudo firewall-cmd --permanent --add-port={6443,2379-2380,10250,10257,10259}/tcp
sudo firewall-cmd --permanent --add-port=8472/udp
sudo firewall-cmd --reload

On each worker node, open the kubelet, kube-proxy and NodePort ranges, plus the VXLAN port:

sudo firewall-cmd --permanent --add-port={10250,10256,30000-32767}/tcp
sudo firewall-cmd --permanent --add-port={8472,30000-32767}/udp
sudo firewall-cmd --reload

Install containerd

The containerd.io package is distributed in Docker's RPM repository. Add the repository and install only the runtime:

sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf -y install containerd.io

The packaged configuration file may have the CRI plugin disabled, so replace it with containerd's full default configuration and switch runc to the systemd cgroup driver, which is the recommended driver on cgroup v2 systems such as the RHEL 9 family:

containerd config default | sudo tee /etc/containerd/config.toml > /dev/null
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
grep SystemdCgroup /etc/containerd/config.toml

The last command must print SystemdCgroup = true. If it prints nothing, add the setting by hand under the runc options table as shown in the container runtimes documentation (the table name differs between containerd 1.x and 2.x). Then start the service:

sudo systemctl enable --now containerd
sudo systemctl restart containerd

Install kubeadm, kubelet and kubectl

Each Kubernetes minor release has its own repository at pkgs.k8s.io. The example uses v1.37, the release the official guide documented when this was written; replace it in both URLs with the minor version you want to install.

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.37/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.37/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF

The exclude line stops a routine dnf update from upgrading Kubernetes behind your back, so the install command has to lift it explicitly:

sudo dnf install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet

The kubelet restarts in a loop until kubeadm gives it a configuration. That is expected.

Initialize the control plane

Run this on the control-plane node only. Flannel's default manifest expects the pod network 10.244.0.0/16, so pass it to kubeadm init:

sudo kubeadm init --pod-network-cidr=10.244.0.0/16

If you may add more control-plane nodes later, also pass --control-plane-endpoint with a DNS name that you can later point at a load balancer; kubeadm cannot convert a cluster created without it to a highly available one.

When the command finishes, it prints a kubeadm join command. Copy it. Then give your regular user access to the cluster:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

The admin.conf file grants full cluster-admin rights. Do not share it.

Install the pod network (CNI)

CoreDNS stays in Pending and nodes stay NotReady until a pod network add-on is installed. Install exactly one. For Flannel, on the control-plane node:

kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

If you chose a different pod CIDR, download the manifest and edit the network in it before applying.

Join the worker nodes

On each worker, run the join command that kubeadm init printed. It has this form:

sudo kubeadm join 203.0.113.10:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>

The initial token expires after 24 hours. To add a node later, generate a fresh command on the control-plane node:

sudo kubeadm token create --print-join-command

Verify the cluster

Back on the control-plane node:

kubectl get nodes -o wide
kubectl get pods --all-namespaces

Within a minute or two every node should be Ready and the CoreDNS pods Running. A running CoreDNS is the sign that the pod network works.

Maintenance and troubleshooting

  • Node stays NotReady: check sudo journalctl -u kubelet -e and kubectl describe node worker1. The usual causes are a missing CNI, swap still active, or a cgroup driver mismatch between kubelet and containerd.
  • Pods on different nodes cannot reach each other: the VXLAN port (UDP 8472) is blocked between nodes, either in firewalld or in your provider's network firewall.
  • Removing a node: drain it, reset it, then delete the node object.
kubectl drain worker1 --delete-emptydir-data --force --ignore-daemonsets
# on worker1:
sudo kubeadm reset
# back on the control-plane node:
kubectl delete node worker1

Upgrades between minor versions are a separate procedure: change the version in kubernetes.repo, then follow the kubeadm upgrade guide one minor release at a time.

Frequently asked questions

Can I still use Docker instead of containerd?

Kubernetes removed its built-in Docker integration, so Docker Engine only works through the separate cri-dockerd adapter. Images built with Docker run unchanged on containerd, so for a cluster node containerd alone is the simpler choice.

Can I run everything on a single server?

Yes. After kubeadm init and the CNI step, remove the control-plane taint with kubectl taint nodes --all node-role.kubernetes.io/control-plane- so ordinary pods can be scheduled on that node. This is fine for testing but gives you no redundancy.

Do I have to leave SELinux permissive?

The kubeadm documentation says you can keep SELinux enforcing if you know how to configure it, but that may need settings kubeadm does not support. Permissive mode is the documented, supported path.

Why does dnf update not upgrade Kubernetes?

The repository file excludes the Kubernetes packages on purpose, and each repository only contains one minor release. Cluster upgrades must go through kubeadm upgrade in a controlled order, control plane first.

Share story:

Continue reading other guides on How VPS.