GitLab not restarting, postresql service not running

After restarting my GitLab server I kept getting the 502 “GitLab is taking too much time to respond” error.

Taking a look at the running services, I get this:

$ sudo gitlab-ctl status

run: gitaly: (pid 1048) 382s; run: log: (pid 1046) 382s

run: gitlab-monitor: (pid 1035) 382s; run: log: (pid 1033) 382s

run: gitlab-workhorse: (pid 1047) 382s; run: log: (pid 1045) 382s

run: logrotate: (pid 1029) 382s; run: log: (pid 1028) 382s

run: nginx: (pid 3900) 15s; run: log: (pid 1026) 382s

run: node-exporter: (pid 1031) 382s; run: log: (pid 1030) 382s

run: postgres-exporter: (pid 1039) 382s; run: log: (pid 1038) 382s

down: postgresql: 0s, normally up, want up; run: log: (pid 1041) 382s

run: prometheus: (pid 3919) 15s; run: log: (pid 1032) 382s

run: redis: (pid 1053) 382s; run: log: (pid 1050) 382s

run: redis-exporter: (pid 1037) 382s; run: log: (pid 1036) 382s

run: sidekiq: (pid 3931) 14s; run: log: (pid 1049) 382s

run: unicorn: (pid 3937) 14s; run: log: (pid 1044) 382s

Everything is up apart from Postgresql. Trying to stop all services and restart, or rebooting the sever still gets the same error. Checking GitLab’s postgresql logs, they show:

2018-03-13_04:04:45.73226 FATAL:  pre-existing shared memory block (key 5432001, ID 0) is still in use

2018-03-13_04:04:45.73232 HINT:  If you're sure there are no old server processes still running, remove the shared memory block or just delete the file "postmaster.pid".

Doing a quick search found this identical question. Following the steps in the first answer:

sudo gitlab-ctl stop
sudo systemctl stop gitlab-runsvdir.service
ps aux | grep postgre (check if there are any postgres processes; shouldn't be)
sudo rm /var/opt/gitlab/postgresql/data/postmaster.pid
sudo systemctl start gitlab-runsvdir.service
sudo gitlab-ctl reconfigure

And then ‘sudo gitlab-ctl start’ and now everything is back up and clean.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.