Posts

Posted on

GitLab CI – docker compose to remote host fails with: Host key verification failed

I’m getting this error when a GitLab CI job is attempting to ‘docker compose up’ to a remote context:

$ docker --context remote compose -f docker-compose-remote-db.yml up -d --pull always
unable to get image '10.0.10.3:5000/my-image:latest': error during connect: Get "http://docker.example.com/v1.51/images/10.0.10.3:5000/adsb-dashboard:latest/json": command [ssh -l gitlab-runner -o ConnectTimeout=30 -T -- 10.0.10.3 docker system dial-stdio] has exited with exit status 255, make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=Host key verification failed.

According to answers here, this could be because when the job runs for the first time the host signature has not been added to known_hosts yet. You can avoid this by:

echo "StrictHostKeyChecking no" >> ~/.ssh/config
Posted on

GitLab Runners stale after SSL cert updates

I’m using Let’s Encrypt SSL certs on my homelab servers which are periodically re-issued using CertBot. This causes my VMs running GitLab Runners to go stale, and when attempting to reverify, they get ‘x509: certificate signed by unknown authority’ errors.

After a cert re-issue, the steps on the GitLab Runner to update it’s certs again I use are:

openssl s_client -showcerts -connect my-homelab-service-name:443 /dev/null|openssl x509 -outform PEM > gitlab.crt

sudo cp gitlab.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
sudo gitlab-runner restart
sudo gitlab-runner verify

After those steps, the Runner status on the GitLab server is back to active and starts picking up pipeline jobs as normal.

Proudly powered by WordPress