Tag: gitlab

Posted on

Building and deploying Docker containers using GitLab CI Pipelines

As part of migrating this blog to Docker containers to move to a different VPS provider (here, here and here), I found myself repeating a number of steps manually, which always a good indication that there’s an opportunity to automate some or all of those steps.

Each time I made a change in the configuration or changed the content to be deployed, I found myself rebuilding the Docker image and either running locally, pushing to my test server, and eventually pushing to my prod VPS and running there.

I’m using a locally running GitLab for my version control, so to use its build pipeline features was a natural next step. I talked about setting up a GitLab runner previously here – this is what performs the work for your pipeline.

You configure your pipeline with a .gitlab-ci.yml file in the root of your repo. I defined 2 stages, build and deploy:

stages:
 - build
 - deploy

For my build stage, I have a single task which is to build my images using my docker-compose.yml:

build:
 stage: build
 script:
 - docker-compose build
 tags:
 - docker-test

For my deploy steps, I defined one for deploying to my test server, and one for deploying to my production VPS. This is the deploy to my locally running Docker server. It changes DOCKER_HOST to point to my test server, and then uses the docker-compose.yml again to bring down the running containers, and bring up the new containers with the updated images:

deploy-containers:
 stage: deploy
 script:
 - export DOCKER_HOST=tcp://192.x.x.x:2375
 - docker-compose down
 - docker-compose up -d
 tags:
 - docker-test

And one for my deploy to production. Note that this step is defined with ‘when: manual’ which tells GitLab the task is only run manually (when you click on the ‘>’ run icon):

prod-deploy-containers:
 stage: deploy
 script:
 - pwd && ls -l
 - ./docker-compose-vps-down.sh
 - ./docker-compose-vps-up.sh
 when: manual
 tags:
 - docker-prod

Here’s what the complete pipeline looks like in GitLab:

With this in place, now any changes committed to the repo result in a new image created and pushed to my test server automatically, and when I’ve completed testing the changes I can optionally deploy the changes to my prod VPS hosted server.

 

 

 

 

Posted on

GitLab not restarting, postresql service not running

After restarting my GitLab server I kept getting the 502 “GitLab is taking too much time to respond” error.

Taking a look at the running services, I get this:

$ sudo gitlab-ctl status

run: gitaly: (pid 1048) 382s; run: log: (pid 1046) 382s

run: gitlab-monitor: (pid 1035) 382s; run: log: (pid 1033) 382s

run: gitlab-workhorse: (pid 1047) 382s; run: log: (pid 1045) 382s

run: logrotate: (pid 1029) 382s; run: log: (pid 1028) 382s

run: nginx: (pid 3900) 15s; run: log: (pid 1026) 382s

run: node-exporter: (pid 1031) 382s; run: log: (pid 1030) 382s

run: postgres-exporter: (pid 1039) 382s; run: log: (pid 1038) 382s

down: postgresql: 0s, normally up, want up; run: log: (pid 1041) 382s

run: prometheus: (pid 3919) 15s; run: log: (pid 1032) 382s

run: redis: (pid 1053) 382s; run: log: (pid 1050) 382s

run: redis-exporter: (pid 1037) 382s; run: log: (pid 1036) 382s

run: sidekiq: (pid 3931) 14s; run: log: (pid 1049) 382s

run: unicorn: (pid 3937) 14s; run: log: (pid 1044) 382s

Everything is up apart from Postgresql. Trying to stop all services and restart, or rebooting the sever still gets the same error. Checking GitLab’s postgresql logs, they show:

2018-03-13_04:04:45.73226 FATAL:  pre-existing shared memory block (key 5432001, ID 0) is still in use

2018-03-13_04:04:45.73232 HINT:  If you're sure there are no old server processes still running, remove the shared memory block or just delete the file "postmaster.pid".

Doing a quick search found this identical question. Following the steps in the first answer:

sudo gitlab-ctl stop
sudo systemctl stop gitlab-runsvdir.service
ps aux | grep postgre (check if there are any postgres processes; shouldn't be)
sudo rm /var/opt/gitlab/postgresql/data/postmaster.pid
sudo systemctl start gitlab-runsvdir.service
sudo gitlab-ctl reconfigure

And then ‘sudo gitlab-ctl start’ and now everything is back up and clean.

Posted on

Assigning GitLab runners to a project

Following on from this post, Runners are added per project (unless they are shared). If you’re configuring a new project and haven’t created a new runner and assigned it to your  project, you’ll see this:

Add a new runner per the steps in the previous article.

Posted on

Configure GitLab to run Docker build in a CI Pipeline

Assuming you already have an installed an running Gitlab server, you need to install and configure a GitLab runner:

Install gitlab-runner: https://docs.gitlab.com/runner/install/linux-repository.html

Run the gitlab-runner shell setup, described here: https://docs.gitlab.com/ce/ci/docker/using_docker_build.html

Get the registration token from the Settings / CI/CD Settings page.

You can now use docker commands in the script section of your .gitlab-ci.yml file defining your pipeline.

Proudly powered by WordPress