Updating Kubernetes master node and worker node config if an ip address changes

I have a test Kubernetes cluster running with a CentOS7 master nodes, and 4 CentOS7 worker nodes, under VMware ESXi. The ip addresses of each of the VMs is from DHCP, and as I hadn’t booted these VMs for a while, when I recently started them up they all got new IP addresses, so the cluster would not start up, and all the .kube/config files were now referring to incorrect IP addresses. Note to self – this is a good reason why you should use DNS names for the nodes in your cluster instead of ip addresses, especially IP addresses that can change.

Anyway, to restore my cluster back to a working state, I reinitialized the master node, and the joined the workers to the new master.

First on the master:

sudo kubeadm reset
sudo kubeadm init --pod-network-cidr=10.244.0.0/16

#take a copy of the kubeadm join command to run on the workers

#copy kube config for local kubectl
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

#apply networking overlay
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.8.0/Documentation/kube-flannel.yml
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.8.0/Documentation/kube-flannel-rbac.yml

#for each of the worker nodes, scp the config file to each node for local kubectl use
scp /etc/kubernetes/admin.conf kev@192.168.1.86:~/.kube/config

On each of the worker nodes:

sudo kubeadm reset

#then run the kubeadm join command shown from the master when you ran kubeadm init

Checking iptables filtering for bridge networking on Ubuntu (for Kubernetes setup)

If you’re installing and configuring a Kubernetes cluster on bare metal or in a VM yourself, one of the install steps using kubeadm says to check iptables filtering for bridge networking, but it doesn’t exactly say how to do this per distro.

The setting required is:

net.bridge.bridge-nf-call-iptables=1

There are specific steps in the kubadm docs above for RHEL/CentOS to add this setting. For Ubuntu it seems this is set by default, but you can confirm by:

sysctl net.bridge.bridge-nf-call-iptables

and the expected setting is 1:

net.bridge.bridge-nf-call-iptables = 1

It seems on Ubuntu 16.04 server this is set to 1 by default, but if it’s 0, you can edit this property in /etc/sysctl.conf

kubernetes: switching kubectl contexts

Info about your currently configured clusters and contexts referring to each of these contexts is stored in ~/.kube/config . You can browse this into with:

kubectl config view

For contexts, scroll down to the contexts section.

For your currently configured context:

 kubectl config current-context

To switch to another context:

kubectl config use-context contextname

Related info on kubectl: https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/

Exposing a public service on Google Kubernetes Engine

To expose a service running on Google Kubernetes Engine to inbound traffic, create a LoadBalancer for the service like this:

kubectl expose deployment exampleservice --type=LoadBalancer --name=exampleservice

Once created, describe the service, and the LoadBalancer Ingress IP is your public IP for the service.

Google’s LoadBalancer service is charged per hour of usage and per GB of traffic. Check the docs for cost.