Revisiting Docker and Kubernetes installation on CentOS7 (take 3)

I tried a while back to get a Kubernetes cluster up and running on CentOS7, and captured my experience in this post here. At one point I did have it up and running, but after a reboot of my master and worker nodes, I ran into an issue with some of the pods not starting, and then decided to shelf this project for a while to work on something else.

Based on tips from a colleague who had recently worked through a similar setup, the main difference in the approach he took compared to my steps was that he didn’t do a vanilla install of Docker with ‘sudo yum install docker’ but instead installed a custom version for CentOS.

Retracing my prior steps, the section in the Kubernetes install instructions here  tell you to do a ‘sudo yum install docker’, but the steps on the Docker site for CentOS here walk you through installing from a custom repo. I followed these steps on a clean CentOS 7 install, and then continued with the Kubernetes setup.

Started the Docker service with:

sudo systemctl start docker.service

Next instead of opening the required ports, since this is just a homelab setup, I just disabled the firewall (following instructions from here):

sudo systemctl disable firewalld

And then stopped it from currently running:

sudo systemctl stop firewalld

Next, picking up with the kubernetes instructions to install kubelet, kubeadm etc.

Disable selinux:

sudo setenforce 0

From my previous steps, editing /etc/selinux/configand setting:

SELINUX=disabled

CentOS7 specific config for iptables (although disabling the firewall on CentOS7 this might not be relevant, but adding it anyway :

cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

Disable swap:

swapoff -a

Also, edit /etc/fstab and remove the swap line and reboot.

Next, following the install instructions to add the repo file, and then installing with:

sudo yum install -y kubelet kubeadm kubectl

Enabling the services:

sudo systemctl enable kubelet && systemctl start kubelet

Starting the node init:

sudo kubeadmin init

And realized we hadn’t addressed the cgroups config issue to get kubelet and docker using the same driver:

Dec 17 18:17:08 unknown000C2954104F kubelet[16450]: error: failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: “systemd” is different from docker cgroup driver: “

I have a post on addressing this with installing Openshift Origin. Follow the same steps here to reconfigure.

Kubeadm init, add the networking overlay (I installed Weave), and I think we’re up:

kubectl get nodes

[kev@unknown000C2954104F /]$ kubectl get nodes

NAME                  STATUS    ROLES     AGE       VERSION

unknown000c2954104f   Ready     master    31m       v1.9.0

Checking the pods though, the dns pod was stuck restarting and not coming up clean. I found this ticket for exactly the issue I was seeing. The resolution was to switch back to cgroupfs for both Docker and Kubernetes.

I did this by backing out the addition previously made to

/usr/lib/systemd/system/docker.service, and then adding a new file, 

/etc/docker/daemon.json, and pasting in:

{
"exec-opts": ["native.cgroupdriver=cgroupfs"]
}

Next, edit /etc/systemd/system/kubelet.service.d/10-kubeadm.conf and replace:

Environment="KUBELET_CGROUP_ARGS=--cgroup-driver=systemd"

with

Environment="KUBELET_CGROUP_ARGS=--cgroup-driver=cgroupfs"

Restart Docker:

sudo systemctl daemon-reload

sudo systemctl restart docker

Check we’re back on cgroupfs:

sudo docker info |grep -i cgroup

Cgroup Driver: cgroupfs

And now check the nodes:

$ kubectl get nodes

NAME                  STATUS    ROLES     AGE       VERSION

unknown000c2954104f   Ready     master    1d        v1.9.0

unknown000c29c2b640   Ready     <none>    1d        v1.9.0

And the pods:

$ kubectl get pods –all-namespaces

NAMESPACE     NAME                                          READY     STATUS    RESTARTS   AGE

kube-system   etcd-unknown000c2954104f                      1/1       Running   3          1d

kube-system   kube-apiserver-unknown000c2954104f            1/1       Running   4          1d

kube-system   kube-controller-manager-unknown000c2954104f   1/1       Running   3          1d

kube-system   kube-dns-6f4fd4bdf-jtk82                      3/3       Running   123        1d

kube-system   kube-proxy-9b6tr                              1/1       Running   3          1d

kube-system   kube-proxy-n5tkx                              1/1       Running   1          1d

kube-system   kube-scheduler-unknown000c2954104f            1/1       Running   3          1d

kube-system   weave-net-f29k9                               2/2       Running   9          1d

kube-system   weave-net-lljgc

 

Now we’re looking good! Next up, lets deploy something and check we’re looking good!

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.