Category: Tech Notes

Posted on

GitLab not restarting, postresql service not running

After restarting my GitLab server I kept getting the 502 “GitLab is taking too much time to respond” error.

Taking a look at the running services, I get this:

$ sudo gitlab-ctl status

run: gitaly: (pid 1048) 382s; run: log: (pid 1046) 382s

run: gitlab-monitor: (pid 1035) 382s; run: log: (pid 1033) 382s

run: gitlab-workhorse: (pid 1047) 382s; run: log: (pid 1045) 382s

run: logrotate: (pid 1029) 382s; run: log: (pid 1028) 382s

run: nginx: (pid 3900) 15s; run: log: (pid 1026) 382s

run: node-exporter: (pid 1031) 382s; run: log: (pid 1030) 382s

run: postgres-exporter: (pid 1039) 382s; run: log: (pid 1038) 382s

down: postgresql: 0s, normally up, want up; run: log: (pid 1041) 382s

run: prometheus: (pid 3919) 15s; run: log: (pid 1032) 382s

run: redis: (pid 1053) 382s; run: log: (pid 1050) 382s

run: redis-exporter: (pid 1037) 382s; run: log: (pid 1036) 382s

run: sidekiq: (pid 3931) 14s; run: log: (pid 1049) 382s

run: unicorn: (pid 3937) 14s; run: log: (pid 1044) 382s

Everything is up apart from Postgresql. Trying to stop all services and restart, or rebooting the sever still gets the same error. Checking GitLab’s postgresql logs, they show:

2018-03-13_04:04:45.73226 FATAL:  pre-existing shared memory block (key 5432001, ID 0) is still in use

2018-03-13_04:04:45.73232 HINT:  If you're sure there are no old server processes still running, remove the shared memory block or just delete the file "postmaster.pid".

Doing a quick search found this identical question. Following the steps in the first answer:

sudo gitlab-ctl stop
sudo systemctl stop gitlab-runsvdir.service
ps aux | grep postgre (check if there are any postgres processes; shouldn't be)
sudo rm /var/opt/gitlab/postgresql/data/postmaster.pid
sudo systemctl start gitlab-runsvdir.service
sudo gitlab-ctl reconfigure

And then ‘sudo gitlab-ctl start’ and now everything is back up and clean.

Posted on

Burning ISOs to disk on MacOS El Cap and later

Prior to El Cap, the Disk Utility on MacOS had an icon to burn an ISO to disk when you mounted the iso. For whatever reason this was removed in El Cap and after, but the ability to burn isos has always bee provided from the Finder.

Right-click an iso file in the Finder and you’ll see a burn option. More info in this article here.

Posted on

kubernetes : kubectl useful commands

Working through the interactive tutorial here is a good reference for kubectl usage.

A few notes for reference:

Copy master node config to a remote machine (from here):

scp root@<master ip>:/etc/kubernetes/admin.conf .

All of the commands if running on a remote machine can use the copied conf file by passing: --kubeconfig ./admin.conf

Query nodes in the cluster:

kubectl get nodes



Show current cluster info:


./kubectl cluster-info

Kubernetes master is running at https://192.168.1.80:6443

KubeDNS is running at https://192.168.1.80:6443/api/v1/namespaces/kube-system/services/kube-dns/proxy


From the interactive tutorial:


Run kubernetes-bootcamp:


kubectl run kubernetes-bootcamp --image=docker.io/jocatalin/kubernetes-bootcamp:v1 --port=8080


Pods:


kubectl get pods
kubectl describe pod podname
kubectl delete pod podname


Deployments:


kubectl get deployments
kubectl describe deployment deploymentname
kubectl delete deployment deploymentname


Get logs


kubectl logs podname

	


	





		

		
Posted on 
Installing Grafana on Ubuntu 16.06
	


	
	

		
Following the install instructions here:


Add the repo to /etc/apt/sources.list and install with apt-get:


deb https://packagecloud.io/grafana/stable/debian/ jessie main


Add key:


curl https://packagecloud.io/gpg.key | sudo apt-key add -


Update and install:


sudo apt-get update
sudo apt-get install grafana


Start up the server service:


sudo service grafana-server start


To start at boot:


sudo update-rc.d grafana-server defaults


Up!



	


	



	
		

	

	





		


		

			

				

	

	

		
		Proudly powered by WordPress