Deploying kubernetes Dashboard to a kubeadm created cluster

Per installations steps here, to deploy the dashboard:

kubectl --kubeconfig ./admin.conf apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

The start the local proxy:

./kubectl --kubeconfig ./admin.conf proxy

Accessing via http://localhost:8001/ui, gives this error:

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "no endpoints available for service \"kubernetes-dashboard\"",
  "reason": "ServiceUnavailable",
  "code": 503
}

Checking what’s currently running with:

./kubectl --kubeconfig ./admin.conf get pods --all-namespaces

Looks like the dashboard app is not happy:

kube-system   kubernetes-dashboard-747c4f7cf-p8blw          0/1       CrashLoopBackOff   22         1h

Checking the logs for the dashboard:

./kubectl --kubeconfig ./admin.conf logs kubernetes-dashboard-747c4f7cf-p8blw --namespace=kube-system

2017/10/19 03:35:51 Error while initializing connection to Kubernetes apiserver. This most likely means that the cluster is misconfigured (e.g., it has invalid apiserver certificates or service accounts configuration) or the –apiserver-host param points to a server that does not exist. Reason: Get https://10.96.0.1:443/version: dial tcp 10.96.0.1:443: getsockopt: no route to host

OK.

I setup my master node using the flannel overlay. I don’t know if this makes any difference or not, but I noticed this article using kubeadm used Weave Net instead. Not knowing how to move forward (and after browsing many posts and tickets on issues with kubeadm with Dashboard), knowing at least that kubadm + Weave Net works for installing dashboard, so I tried this approach instead.

After re-initializing and the adding weave-net, my pods are all started:

$ kubectl get pods –all-namespaces

NAMESPACE     NAME                                          READY     STATUS    RESTARTS   AGE

kube-system   etcd-unknown000c2960f639                      1/1       Running   0          11m

kube-system   kube-apiserver-unknown000c2960f639            1/1       Running   0          11m

kube-system   kube-controller-manager-unknown000c2960f639   1/1       Running   0          11m

kube-system   kube-dns-545bc4bfd4-nhrw7                     3/3       Running   0          12m

kube-system   kube-proxy-cgn45                              1/1       Running   0          4m

kube-system   kube-proxy-dh6jm                              1/1       Running   0          12m

kube-system   kube-proxy-spxm5                              1/1       Running   0          5m

kube-system   kube-scheduler-unknown000c2960f639            1/1       Running   0          11m

kube-system   weave-net-gs8nh                               2/2       Running   0          5m

kube-system   weave-net-jkkql                               2/2       Running   0          4m

kube-system   weave-net-xb4hx                               2/2       Running   0          10m

 

Trying to add the dashboard once more:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

… and, o. m. g. :

$ kubectl get pods –all-namespaces

NAMESPACE     NAME                                          READY     STATUS    RESTARTS   AGE

kube-system   etcd-unknown000c2960f639                      1/1       Running   0          37m

kube-system   kube-apiserver-unknown000c2960f639            1/1       Running   0          37m

kube-system   kube-controller-manager-unknown000c2960f639   1/1       Running   0          37m

kube-system   kube-dns-545bc4bfd4-nhrw7                     3/3       Running   0          38m

kube-system   kube-proxy-cgn45                              1/1       Running   0          30m

kube-system   kube-proxy-dh6jm                              1/1       Running   0          38m

kube-system   kube-proxy-spxm5                              1/1       Running   0          31m

kube-system   kube-scheduler-unknown000c2960f639            1/1       Running   0          37m

kube-system   kubernetes-dashboard-747c4f7cf-jgmgt          1/1       Running   0          4m

kube-system   weave-net-gs8nh                               2/2       Running   0          31m

kube-system   weave-net-jkkql                               2/2       Running   0          30m

kube-system   weave-net-xb4hx                               2/2       Running   0          36m

Starting kubectl proxy and hitting localhost:8001/ui now gives me:

Error: 'malformed HTTP response "\x15\x03\x01\x00\x02\x02"'
Trying to reach: 'http://10.32.0.3:8443/'

Reading here, trying the master node directly:

https://192.168.1.80:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/

gives a different error:

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "services \"https:kubernetes-dashboard:\" is forbidden: User \"system:anonymous\" cannot get services/proxy in the namespace \"kube-system\"",
  "reason": "Forbidden",
  "details": {
    "name": "https:kubernetes-dashboard:",
    "kind": "services"
  },
  "code": 403
}

… but reading further ahead, it seems accessing via the /ui url is not correctly working, you need to access via the url in the docs here,  which says the correct url is:

http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/

and now I get an authentication page:

Time to read ahead on the authentication approaches.

List available tokens with:

kubectl -n kube-system get secret

Using the same token as per the docs (although at this point I’ve honestly no idea what the difference in permissions is for each of the different tokens):

./kubectl --kubeconfig admin.conf -n kube-system describe secret replicaset-controller-token-7tzd5

And then pasting the token value into the authentication dialog gets me logged on! There’s some errors about this token not having access to some features, but at this point I’ve just glad I’ve managed to get this deployed and working!

If you’re intested in the specific versions I’m using, this is deployed to CentOS 7, and kubernetes version:

$ kubectl version

Client Version: version.Info{Major:”1″, Minor:”8″, GitVersion:”v1.8.1″, GitCommit:”f38e43b221d08850172a9a4ea785a86a3ffa3b3a”, GitTreeState:”clean”, BuildDate:”2017-10-11T23:27:35Z”, GoVersion:”go1.8.3″, Compiler:”gc”, Platform:”linux/amd64″}

Server Version: version.Info{Major:”1″, Minor:”8″, GitVersion:”v1.8.1″, GitCommit:”f38e43b221d08850172a9a4ea785a86a3ffa3b3a”, GitTreeState:”clean”, BuildDate:”2017-10-11T23:16:41Z”, GoVersion:”go1.8.3″, Compiler:”gc”, Platform:”linux/amd64″}

Installing Java 7 JDK on CentOS 7

Openjdk packages on CentOS 7 come in a JRE and a JDK (as you’d expect). If you install java-1.7.0-openjdk you get just the JRE:

$ sudo yum install java-1.7.0-openjdk

$ java -version

java version "1.7.0_151"

OpenJDK Runtime Environment (rhel-2.6.11.1.el7_4-x86_64 u151-b00)

OpenJDK 64-Bit Server VM (build 24.151-b00, mixed mode)

Starting up WLS 10.3.6:

wlserver_10.3]$ cd server/

$ cd bin

$ . ./setWLSEnv.sh

The JDK wasn’t found in directory /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.151-2.6.11.1.el7_4.x86_64.

Please edit the startWebLogic.sh script so that the JAVA_HOME

variable points to the location of your JDK.

$ cd /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.151-2.6.11.1.el7_4.x86_64

$ ls

ASSEMBLY_EXCEPTION  jre  jre-abrt  LICENSE  THIRD_PARTY_README

Only a JRE there, no JDK.

From here, install the devel package too:

sudo yum install java-1.7.0-openjdk-devel

Now we look good:

$ . ./setWLSEnv.sh

CLASSPATH=/home/kev/Oracle/Middleware/patch_wls1036/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/kev/Oracle/Middleware/patch_ocp371/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.151-2.6.11.1.el7_4.x86_64/lib/tools.jar:/home/kev/Oracle/Middleware/wlserver_10.3/server/lib/weblogic_sp.jar:/home/kev/Oracle/Middleware/wlserver_10.3/server/lib/weblogic.jar:/home/kev/Oracle/Middleware/modules/features/weblogic.server.modules_10.3.6.0.jar:/home/kev/Oracle/Middleware/wlserver_10.3/server/lib/webservices.jar:/home/kev/Oracle/Middleware/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/home/kev/Oracle/Middleware/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:

PATH=/home/kev/Oracle/Middleware/wlserver_10.3/server/bin:/home/kev/Oracle/Middleware/modules/org.apache.ant_1.7.1/bin:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.151-2.6.11.1.el7_4.x86_64/jre/bin:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.151-2.6.11.1.el7_4.x86_64/bin:/home/kev/Oracle/Middleware/wlserver_10.3/server/bin:/home/kev/Oracle/Middleware/modules/org.apache.ant_1.7.1/bin:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.151-2.6.11.1.el7_4.x86_64/jre/bin:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.151-2.6.11.1.el7_4.x86_64/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/kev/.local/bin:/home/kev/bin

Your environment has been set.

Good to go!

Detecting attached monitors on MacOS

If you boot MacOS with a KVM attached and a second monitor is not actually attached at boot time, on my 2008 Mac Pro at least, the second monitor is not automatically detected (it works fine if the KVM is already switched to the Mac at boot).

To force MacOS to detect additional monitors, hold the Option key in System Preferences / Displays, and then click on the ‘Detect Monitors’ button that appears.

See article here.

Cloning an existing VM on VMware ESXi using command line vmkfstools

Apparently vCenter Server provides the ability to clone VMs via the Client, but not if you’re just using ESXi and managing your host directly with the web client. It is possible however to clone a VM’s disk using the vmkfstools commanline utility as described in this post.

Here’s a summary of the steps:

Enable SSH from the ESXi web console: Host / Manage / Services

In my case I wanted to create a copy of an existing CentOS7 VM. SSH into your ESXi host, then:

vmkfstools -i CentOS7-1/CentOS7-1_0.vmdk CentOS7-2/CentOS7-2.vmdk -d thin

Next, create a new VM as normal, but on the Customize Settings dialog, press the X on the right to delete the disk created by the new VM wizard:

Next, press ‘Add new disk’, select ‘Existing hard disk’, then point to the copy of the VM disk that you created with the vmkfstools command:

Credit to this post for the tip to configure using an existing disk.