Migration to new VPS running my blog in Docker containers now complete!

After many more hours than I expected or planned, I’ve migrated this site to run on a new VPS provider running in a larger KVM based VPS. The site is now running with nginx and php5-fpm in one Docker container, and MySQL in another, linked together with docker-compose.

Along the way I ran into several issues around performance and firewall configurations, which led to setting up a GitLab CI/CD pipeline (here and here) so I could more quickly iterate and deploy changes to a local test VM server on my ESXi rack server. I set up this test VM to mirror the configuration in my VPS KVM, and then used a GitLab pipeline to push the containers to my test server, and then manually push to my production VPS server when ready to deploy.

The good news is I learned plenty along the way, but also went down several rabbit holes trying to chase down performance issues that turned out to be more related to my misconfiguration of Ubuntu’s UFW and Dockers interaction with iptables that caused some weirdness.

The other good news is I have plenty of RAM and CPU to spare in this KVM based VPS where I’m running Docker, so I’ll be able to take advantage of this to deploy some other projects too (this was one of my other reasons for migrating to another server/provider). I’ll share some additional posts about some of the specifics of the GitLab CI/CD config, dockerfile and docker-compose configurations in the next few days.

Enabling Docker service to listen on a port

By default the Docker service listens on a local socket. If you want to access the Docker service api remotely, you need to configure the service to listen on a port as well.

On Ubuntu 16.04, edit /lib/systemd/system/docker.service and change this line:

ExecStart=/usr/bin/dockerd -H fd://

to

ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2376

Reload the systemd config:

sudo systemctl daemon-reload

and restart the service:

sudo systemctl restart docker.service

More info here.

Ubuntu 14.04 Dockerd : unknown option dirperm1 on startup

Trying to start the Dockerd service on an Ubuntu 14.04 server I ran into ‘sudo service docker start’ hanging, and in the logs seeing this message:

Mar  4 17:12:07 vps kernel: [58270.204343] aufs au_opts_parse:1155:dockerd[12023]: unknown option dirperm1

Mar  4 17:12:49 vps kernel: [58311.799010] init: docker post-start process (12015) terminated with status 1

This is described in this post here and here.

In the Docker CE for Ubuntu instructions, there’a a note for installing additional aufs support for 14.04 here:

sudo apt-get install \
    linux-image-extra-$(uname -r) \
    linux-image-extra-virtual

Note the Docker CE for Ubuntu install docs say for 16.04:

For Ubuntu 16.04 and higher, the Linux kernel includes support for OverlayFS, and Docker CE uses the overlay2 storage driver by default.

In this particular case I had mistakenly setup this VM from a 14.04 template, and I really wanted to be running 16.04. With 16.04 and the latest Docker install per the installed steps here, I didn’t run into this issue.