Module 2 — Cluster Installation with kubeadm¶

Overview¶

kubeadm is the official Kubernetes tool for bootstrapping clusters. The CKA exam expects you to understand the full installation process: prerequisites, kubeadm init, kubeadm join, installation phases, CNI plugin deployment, and high-availability topologies.

1. Prerequisites¶

Before running kubeadm, every node (control plane and workers) must meet these requirements:

1.1 System Requirements¶

Requirement	Control Plane	Worker
CPU	2+ cores	1+ core
RAM	2 GB+	1 GB+
Disk	10 GB+	10 GB+
OS	Ubuntu 22.04 / RHEL 8+ / Debian 12	Same
Swap	Disabled	Disabled
Unique hostname	Yes	Yes
Unique MAC / product_uuid	Yes	Yes

1.2 Disable Swap¶

# Disable swap immediately
swapoff -a

# Disable swap permanently (comment out swap entries)
sed -i '/ swap / s/^/#/' /etc/fstab

1.3 Load Required Kernel Modules¶

cat <<EOF | tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

modprobe overlay
modprobe br_netfilter

1.4 Set Sysctl Parameters¶

cat <<EOF | tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

sysctl --system

1.5 Install a Container Runtime (containerd)¶

# Install containerd
apt-get update && apt-get install -y containerd

# Generate default config
mkdir -p /etc/containerd
containerd config default | tee /etc/containerd/config.toml

# Enable SystemdCgroup (required for kubeadm)
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

# Restart containerd
systemctl restart containerd
systemctl enable containerd

Why SystemdCgroup? kubelet uses systemd as the cgroup driver by default. The container runtime must match. Mismatched cgroup drivers cause kubelet to fail.

1.6 Install kubeadm, kubelet, and kubectl¶

# Add Kubernetes apt repository (v1.30 example)
apt-get update && apt-get install -y apt-transport-https ca-certificates curl gpg

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | \
  gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | \
  tee /etc/apt/sources.list.d/kubernetes.list

# Install
apt-get update
apt-get install -y kubelet kubeadm kubectl

# Pin versions to prevent accidental upgrades
apt-mark hold kubelet kubeadm kubectl

1.7 Required Ports¶

Component	Ports	Protocol
kube-apiserver	6443	TCP
etcd	2379-2380	TCP
kubelet	10250	TCP
kube-scheduler	10259	TCP
kube-controller-manager	10257	TCP
NodePort Services	30000-32767	TCP

# Verify ports are open (on each node)
nc -zv <control-plane-ip> 6443

2. Initializing the Control Plane — kubeadm init¶

2.1 Basic Init¶

kubeadm init \
  --pod-network-cidr=10.244.0.0/16 \
  --apiserver-advertise-address=192.168.1.10 \
  --kubernetes-version=v1.30.0

Flag	Purpose
`--pod-network-cidr`	CIDR for the pod network — must match your CNI plugin's expected range
`--apiserver-advertise-address`	IP the API server advertises to other components
`--kubernetes-version`	Specific K8s version to install
`--control-plane-endpoint`	Stable endpoint for HA setups (DNS name or load balancer IP)
`--service-cidr`	CIDR for Services (default: `10.96.0.0/12`)
`--cri-socket`	Container runtime socket (auto-detected if only one runtime is installed)

2.2 What kubeadm init Does (Phases)¶

kubeadm init runs through a series of phases in order:

kubeadm init
  │
  ├── preflight          ← Validates system requirements
  ├── certs              ← Generates all PKI certificates
  │   ├── ca
  │   ├── apiserver
  │   ├── apiserver-kubelet-client
  │   ├── front-proxy-ca
  │   ├── front-proxy-client
  │   ├── etcd/ca
  │   ├── etcd/server
  │   ├── etcd/peer
  │   ├── etcd/healthcheck-client
  │   ├── apiserver-etcd-client
  │   └── sa (service account key pair)
  ├── kubeconfig         ← Generates kubeconfig files
  │   ├── admin.conf
  │   ├── kubelet.conf
  │   ├── controller-manager.conf
  │   └── scheduler.conf
  ├── kubelet-start       ← Writes kubelet config and starts kubelet
  ├── control-plane       ← Creates static pod manifests
  │   ├── apiserver
  │   ├── controller-manager
  │   └── scheduler
  ├── etcd               ← Creates etcd static pod manifest
  ├── upload-config      ← Uploads kubeadm and kubelet config to ConfigMaps
  ├── upload-certs       ← (HA only) Uploads certs to a Secret
  ├── mark-control-plane ← Adds labels and taints to the control plane node
  ├── bootstrap-token    ← Creates bootstrap token for joining nodes
  └── addon              ← Installs CoreDNS and kube-proxy

CKA Tip: You can run individual phases with kubeadm init phase <phase-name>. This is useful for troubleshooting or custom setups.

# Example: regenerate certificates only
kubeadm init phase certs all

# Example: generate only the API server certificate
kubeadm init phase certs apiserver

# List all available phases
kubeadm init phase --help

2.3 After kubeadm init¶

The output gives you three critical pieces of information:

# 1. Set up kubectl for the admin user
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

# 2. Install a CNI plugin (covered in Section 4)
kubectl apply -f <cni-manifest-url>

# 3. Join command for worker nodes (save this!)
kubeadm join 192.168.1.10:6443 --token abcdef.0123456789abcdef \
  --discovery-token-ca-cert-hash sha256:abc123...

2.4 Using a Configuration File¶

Instead of passing flags, you can use a YAML configuration file:

# kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: v1.30.0
controlPlaneEndpoint: "k8s-lb.example.com:6443"
networking:
  podSubnet: "10.244.0.0/16"
  serviceSubnet: "10.96.0.0/12"
  dnsDomain: "cluster.local"
apiServer:
  certSANs:
    - "k8s-lb.example.com"
    - "192.168.1.100"
etcd:
  local:
    dataDir: /var/lib/etcd
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: "192.168.1.10"
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock

kubeadm init --config kubeadm-config.yaml

CKA Tip: You can print the default configuration with:
1
2
kubeadm config print init-defaults
kubeadm config print join-defaults

3. Joining Worker Nodes — kubeadm join¶

3.1 Using the Join Command¶

# On each worker node (use the command from kubeadm init output)
kubeadm join 192.168.1.10:6443 \
  --token abcdef.0123456789abcdef \
  --discovery-token-ca-cert-hash sha256:abc123...

3.2 What kubeadm join Does¶

kubeadm join
  │
  ├── preflight          ← Validates system requirements
  ├── discovery          ← Contacts API server, validates CA cert hash
  ├── kubelet-start      ← Configures and starts kubelet
  └── (optional) control-plane  ← If --control-plane flag is used

3.3 Token Management¶

Tokens expire after 24 hours by default.

# List existing tokens
kubeadm token list

# Create a new token
kubeadm token create

# Create a new token and print the full join command
kubeadm token create --print-join-command

# Get the CA cert hash (if you lost it)
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | \
  openssl rsa -pubin -outform der 2>/dev/null | \
  openssl dgst -sha256 -hex | sed 's/^.* //'

CKA Tip: kubeadm token create --print-join-command is the fastest way to get a working join command. Memorize this.

3.4 Verify Nodes Joined¶

kubectl get nodes
# NAME           STATUS     ROLES           AGE   VERSION
# controlplane   NotReady   control-plane   5m    v1.30.0
# worker-1       NotReady   <none>          2m    v1.30.0
# worker-2       NotReady   <none>          1m    v1.30.0

Nodes will show NotReady until a CNI plugin is installed.

4. Choosing and Installing a CNI Plugin¶

4.1 Why CNI Is Required¶

Kubernetes does not ship with a built-in network implementation. The Container Network Interface (CNI) plugin provides:

Pod-to-pod networking across nodes
IP address management (IPAM) for pods
Network policy enforcement (plugin-dependent)

Without a CNI plugin: - Nodes remain NotReady - CoreDNS pods stay in Pending state - No pod-to-pod communication is possible

4.2 CNI Plugin Comparison¶

Feature	Calico	Flannel	Cilium
Network Policy	✅ Full support	❌ None	✅ Full + extended
Encryption	WireGuard	None	WireGuard / IPsec
Routing	BGP, VXLAN, IP-in-IP	VXLAN	VXLAN, native routing
Performance	High	Moderate	High (eBPF)
Complexity	Medium	Low	High
CKA relevance	Most commonly tested	Simple labs	Growing adoption

CKA Tip: The exam typically uses Calico or Flannel. The install command is usually provided in the exam question or you can find it in the official docs.

4.3 Installing Calico¶

# Calico with default settings (pod CIDR: 192.168.0.0/16)
kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml

If you used --pod-network-cidr=10.244.0.0/16 during init, modify the Calico manifest:

# Download the manifest
curl -O https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml

# Edit the CALICO_IPV4POOL_CIDR to match your pod CIDR
# Find and uncomment/modify:
#   - name: CALICO_IPV4POOL_CIDR
#     value: "10.244.0.0/16"

kubectl apply -f calico.yaml

4.4 Installing Flannel¶

# Flannel expects --pod-network-cidr=10.244.0.0/16
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

Important: Flannel's default CIDR is 10.244.0.0/16. Your --pod-network-cidr must match.

4.5 Installing Cilium¶

# Using Helm (preferred)
helm repo add cilium https://helm.cilium.io/
helm install cilium cilium/cilium --namespace kube-system

# Or using the Cilium CLI
cilium install

4.6 Verifying CNI Installation¶

# Nodes should become Ready
kubectl get nodes
# NAME           STATUS   ROLES           AGE   VERSION
# controlplane   Ready    control-plane   10m   v1.30.0
# worker-1       Ready    <none>          7m    v1.30.0

# CoreDNS pods should be Running
kubectl get pods -n kube-system | grep coredns

# Check CNI configuration on a node
ls /etc/cni/net.d/
cat /etc/cni/net.d/*.conflist

# Check CNI binaries
ls /opt/cni/bin/

# Test pod-to-pod connectivity
kubectl run test1 --image=busybox --command -- sleep 3600
kubectl run test2 --image=busybox --command -- sleep 3600
kubectl exec test1 -- ping -c 3 $(kubectl get pod test2 -o jsonpath='{.status.podIP}')

4.7 CNI Troubleshooting¶

Symptom	Likely Cause	Fix
Nodes stuck in `NotReady`	CNI not installed	Install a CNI plugin
CoreDNS pods `Pending`	No CNI available	Install a CNI plugin
Pods can't communicate across nodes	CIDR mismatch	Ensure `--pod-network-cidr` matches CNI config
CNI pods `CrashLoopBackOff`	Missing kernel modules	Load `overlay` and `br_netfilter` modules
Pods get IP but can't reach other pods	Firewall blocking VXLAN (UDP 4789) or IP-in-IP (protocol 4)	Open required ports between nodes

5. kubeadm init Phases — Deep Dive¶

Understanding phases is critical for troubleshooting and for the exam.

5.1 Preflight Checks¶

# Run preflight checks without actually initializing
kubeadm init phase preflight

# Common preflight failures:
# - [ERROR Swap]: swap is enabled
# - [ERROR Port-6443]: port 6443 is in use
# - [ERROR FileAvailable--etc-kubernetes-manifests-*]: manifest already exists
# - [ERROR NumCPU]: the number of CPUs is less than required

5.2 Certificate Generation¶

kubeadm generates a full PKI under /etc/kubernetes/pki/:

/etc/kubernetes/pki/
├── ca.crt                    # Cluster CA certificate
├── ca.key                    # Cluster CA private key
├── apiserver.crt             # API server serving certificate
├── apiserver.key
├── apiserver-kubelet-client.crt
├── apiserver-kubelet-client.key
├── apiserver-etcd-client.crt
├── apiserver-etcd-client.key
├── front-proxy-ca.crt        # Front proxy CA
├── front-proxy-ca.key
├── front-proxy-client.crt
├── front-proxy-client.key
├── sa.key                    # Service account signing key
├── sa.pub                    # Service account verification key
└── etcd/
    ├── ca.crt                # etcd CA certificate
    ├── ca.key
    ├── server.crt
    ├── server.key
    ├── peer.crt
    ├── peer.key
    ├── healthcheck-client.crt
    └── healthcheck-client.key

# Check certificate expiration
kubeadm certs check-expiration

# Renew all certificates
kubeadm certs renew all

# Renew a specific certificate
kubeadm certs renew apiserver

CKA Tip: Certificates are valid for 1 year by default. The CA certificate is valid for 10 years.

5.3 Static Pod Manifests¶

After init, control plane components run as static pods:

ls /etc/kubernetes/manifests/
# etcd.yaml
# kube-apiserver.yaml
# kube-controller-manager.yaml
# kube-scheduler.yaml

kubelet watches this directory and automatically creates/restarts these pods. If you modify a manifest, kubelet detects the change and recreates the pod.

6. High-Availability Control Plane¶

6.1 Why HA?¶

A single control plane node is a single point of failure. If it goes down: - No new pods can be scheduled - No scaling or self-healing - No API access (kubectl stops working) - Existing workloads continue running on worker nodes (kubelet keeps them alive)

6.2 HA Topologies¶

kubeadm supports two HA topologies:

Stacked etcd (Recommended for simplicity)¶

etcd runs on the same nodes as the control plane components.

          +----------------------+
          |    Load Balancer     |
          |  k8s-lb:6443         |
          +----------+-----------+
                     |
     +---------------+---------------+
     |               |               |
+----+------+   +----+------+   +----+------+
| CP Node1  |   | CP Node2  |   | CP Node3  |
|           |   |           |   |           |
| apiserver |   | apiserver |   | apiserver |
| scheduler |   | scheduler |   | scheduler |
| ctrl-mgr  |   | ctrl-mgr  |   | ctrl-mgr  |
| etcd      |   | etcd      |   | etcd      |
+-----------+   +-----------+   +-----------+

Pros: Fewer nodes, simpler setup
Cons: Losing a node loses both a control plane member and an etcd member

External etcd¶

etcd runs on dedicated nodes, separate from the control plane.

        ┌──────────────────────┐
        │    Load Balancer     │
        │  k8s-lb:6443         │
        └──────────┬───────────┘
                   │
     ┌─────────────┼─────────────┐
     │             │             │
┌────▼────┐  ┌────▼────┐  ┌────▼────┐
│ CP Node1│  │ CP Node2│  │ CP Node3│
│         │  │         │  │         │
│ apiserver│  │ apiserver│  │ apiserver│
│ scheduler│  │ scheduler│  │ scheduler│
│ ctrl-mgr │  │ ctrl-mgr │  │ ctrl-mgr │
└─────────┘  └─────────┘  └─────────┘

┌─────────┐  ┌─────────┐  ┌─────────┐
│ etcd-1  │  │ etcd-2  │  │ etcd-3  │
└─────────┘  └─────────┘  └─────────┘

Pros: etcd failure doesn't affect control plane, independent scaling
Cons: More nodes to manage, more complex setup

6.3 Load Balancer Requirement¶

HA requires a load balancer in front of the API servers:

Option	Type	Notes
HAProxy + keepalived	On-premise	Most common for bare-metal
Cloud LB (NLB/ALB)	Cloud	AWS, Azure, GCP native LBs
kube-vip	Kubernetes-native	Virtual IP, runs as static pod

Example HAProxy configuration:

frontend k8s-api
    bind *:6443
    mode tcp
    default_backend k8s-api-backend

backend k8s-api-backend
    mode tcp
    balance roundrobin
    option tcp-check
    server cp1 192.168.1.10:6443 check
    server cp2 192.168.1.11:6443 check
    server cp3 192.168.1.12:6443 check

6.4 Setting Up HA with kubeadm¶

Step 1 — Initialize the first control plane node¶

kubeadm init \
  --control-plane-endpoint "k8s-lb.example.com:6443" \
  --upload-certs \
  --pod-network-cidr=10.244.0.0/16

--control-plane-endpoint — points to the load balancer (critical for HA)
--upload-certs — encrypts and uploads certificates to a kubeadm-certs Secret

Step 2 — Join additional control plane nodes¶

# The init output provides this command:
kubeadm join k8s-lb.example.com:6443 \
  --token abcdef.0123456789abcdef \
  --discovery-token-ca-cert-hash sha256:abc123... \
  --control-plane \
  --certificate-key <certificate-key>

--control-plane — tells kubeadm this node is a control plane member
--certificate-key — decryption key for the uploaded certificates (valid for 2 hours)

Step 3 — Join worker nodes¶

kubeadm join k8s-lb.example.com:6443 \
  --token abcdef.0123456789abcdef \
  --discovery-token-ca-cert-hash sha256:abc123...

6.5 etcd Quorum¶

etcd uses the Raft consensus algorithm and requires a majority (quorum) to function:

etcd Members	Quorum	Tolerated Failures
1	1	0
2	2	0 (worse than 1!)
3	2	1
5	3	2
7	4	3

Key insight: Always use an odd number of etcd members. 2 members is worse than 1 because you still need both for quorum but have more points of failure.

6.6 Leader Election¶

In an HA setup, only one instance of the scheduler and controller manager is active at a time. The others are on standby using leader election:

# Check which node holds the scheduler lease
kubectl get lease kube-scheduler -n kube-system -o yaml

# Check which node holds the controller-manager lease
kubectl get lease kube-controller-manager -n kube-system -o yaml

The API server, however, runs active-active — all instances serve requests simultaneously behind the load balancer.

7. Resetting a Cluster¶

If you need to start over:

# On each node (control plane and workers)
kubeadm reset

# Clean up CNI configuration
rm -rf /etc/cni/net.d

# Clean up iptables rules
iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X

# (Optional) Clean up IPVS rules
ipvsadm --clear

# Remove kubeconfig
rm -rf $HOME/.kube/config

8. Practice Exercises¶

Exercise 1 — Install a Cluster from Scratch¶

# On ALL nodes:
# 1. Disable swap
# 2. Load kernel modules (overlay, br_netfilter)
# 3. Set sysctl parameters
# 4. Install containerd with SystemdCgroup = true
# 5. Install kubeadm, kubelet, kubectl

# On the control plane node:
# 6. Run kubeadm init with --pod-network-cidr=10.244.0.0/16
# 7. Set up kubectl
# 8. Install Calico or Flannel
# 9. Verify nodes are Ready

# On each worker node:
# 10. Run kubeadm join with the token from step 6
# 11. Verify all nodes are Ready from the control plane

Exercise 2 — Explore kubeadm Phases¶

# 1. Print the default init configuration
kubeadm config print init-defaults

# 2. Run only the preflight checks
kubeadm init phase preflight --config kubeadm-config.yaml

# 3. Check certificate expiration dates
kubeadm certs check-expiration

# 4. List all certificates in /etc/kubernetes/pki/
find /etc/kubernetes/pki/ -name "*.crt" -exec openssl x509 -in {} -noout -subject -dates \;

Exercise 3 — Token Management¶

# 1. List all bootstrap tokens
kubeadm token list

# 2. Create a new token with a 1-hour TTL
kubeadm token create --ttl 1h

# 3. Generate a full join command
kubeadm token create --print-join-command

# 4. Manually compute the CA cert hash
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | \
  openssl rsa -pubin -outform der 2>/dev/null | \
  openssl dgst -sha256 -hex | sed 's/^.* //'

Exercise 4 — Break and Fix¶

# Scenario 1: Wrong pod CIDR
# - Initialize a cluster with --pod-network-cidr=10.244.0.0/16
# - Install Calico with default CIDR (192.168.0.0/16)
# - Observe: pods get IPs from 192.168.x.x, not 10.244.x.x
# - Fix: modify the Calico manifest to use 10.244.0.0/16

# Scenario 2: Missing CNI
# - After kubeadm init, don't install a CNI
# - Observe: nodes are NotReady, CoreDNS is Pending
# - Fix: install a CNI plugin

# Scenario 3: Expired token
# - Try to join a worker node with an expired token
# - Observe: "token has expired" error
# - Fix: kubeadm token create --print-join-command

9. Key Takeaways for the CKA Exam¶

Point	Detail
Know the prerequisites	Swap off, kernel modules, sysctl params, container runtime, kubeadm/kubelet/kubectl
kubeadm init phases	Understand what each phase does — you may need to run individual phases
--pod-network-cidr must match CNI	Flannel expects `10.244.0.0/16`, Calico defaults to `192.168.0.0/16`
Tokens expire in 24h	Use `kubeadm token create --print-join-command` to regenerate
HA requires a load balancer	`--control-plane-endpoint` must point to the LB, not a single node
Stacked vs external etcd	Know both topologies and their trade-offs
Odd number of etcd members	3 or 5 — never 2
Leader election	Scheduler and controller manager use leases; API server is active-active
kubeadm reset	Cleans up a node — remember to also clean CNI config and iptables
Config file approach	`kubeadm init --config` is cleaner than many CLI flags

Next: 03-cluster-upgrade.md — Upgrading a cluster with kubeadm