Migration to new VPS running my blog in Docker containers now complete!

After many more hours than I expected or planned, I’ve migrated this site to run on a new VPS provider running in a larger KVM based VPS. The site is now running with nginx and php5-fpm in one Docker container, and MySQL in another, linked together with docker-compose.

Along the way I ran into several issues around performance and firewall configurations, which led to setting up a GitLab CI/CD pipeline (here and here) so I could more quickly iterate and deploy changes to a local test VM server on my ESXi rack server. I set up this test VM to mirror the configuration in my VPS KVM, and then used a GitLab pipeline to push the containers to my test server, and then manually push to my production VPS server when ready to deploy.

The good news is I learned plenty along the way, but also went down several rabbit holes trying to chase down performance issues that turned out to be more related to my misconfiguration of Ubuntu’s UFW and Dockers interaction with iptables that caused some weirdness.

The other good news is I have plenty of RAM and CPU to spare in this KVM based VPS where I’m running Docker, so I’ll be able to take advantage of this to deploy some other projects too (this was one of my other reasons for migrating to another server/provider). I’ll share some additional posts about some of the specifics of the GitLab CI/CD config, dockerfile and docker-compose configurations in the next few days.

nginx + php5-fpm response lag on first requests

I’m in the middle of migrating this existing site to Docker containers and moving to a new VPS host. Part of my motivation for the move is to capture the customized configs for each of the servers, so I can easily move the whole deployment between a test environment and a production deploy. What’s prompting this is the realization that the majority of the performance tweaks I made during the first native install I have captured in various blog posts here, but to recreate these install steps I would need to go back to each of those articles and get the details in order to repeat them elsewhere. That’s not a particularly repeatable process.

I’m close to switch from my current non-Docker install (here, as of 2/27/18) to my test install now running on Docker. I’ll share more about how that is configured in future posts, but I just wanted to capture one nginx + php5-fpm specific config that had me stumped for a few days.

There’s many options for configuring the worker processes for nginx and php5-fpm. php5-fpm itself has a number of modes that control how it manages it’s worker processes. By default the process manager is ‘dynamic’ (pm = dynamic). This creates processes to handle incoming requests based on the other related config options (max_children, start servers, min_spare_servers, max_spare_servers etc).

On my current site based on recommendations I changed this to pm = ondemand in order to minimize memory usage on my 512MB VPS. One other param though had an interesting effect:

pm.process_idle_timeout = 10s;

This keeps a process alive for an additional 10s after it’s finished the current request. This seems to have an impact on the responsiveness of the WordPress site, as without this there seems to be a noticeable lag of 3-4 seconds before responses start to come back to the browser, presumably because new worker processes are needed to restart to handle the next request – by keeping them up after the last request there no lag to restart a new process.

I was almost at the point of making a no-go decision based on the laggy performance, but adding this one param has fixed the laggy behavior, and now I’m looking all set. Given that I’ve jumped from a 512MB VPS to a 4GB VPS, I’m less concerned about keeping memory usage to a minimum this time so I haven’t changed from dynamic to ondemand in the Docker config for my new nginx + php5-fpm config, but this one param is worth knowing about.

Moving my nginx+mysql WordPress VPS native install to Docker containers on a KVM VPS

My WordPress blog that you’re reading right now is running on nginx and MySQL installed on a cheap OpenVZ VPS. I’ve been running on a $2.50 VPS from Virmach for the past 6 months or so and been very happy with the service. I spent a bunch of time tweaking the nginx and MySQL config params to run in < 512MB, which it does comfortably, but nginx and MySQL are both installed directly on the Ubuntu VM instance and it would be great of I could make this setup more easily movable between cloud providers (or even to have a local copy of the setup for testing, vs the live site).

I’ve been spending a lot of time playing with Docker and Kubernetes, so it seems logical that I should move the site into containers and then this will allow me to explore other deployment options.

Migration Steps – find a KVM VPS

As far as I know you can’t install Docker in an OpenVZ virtualized VPS container, so first step I need to move to a KVM based VPS so I can install Docker (and possibly Kubernetes). I’ve been shopping the deals on lowendbox.com and there’s plenty of reasonably deals for around $5/month for various combinations of 2 to 4GB RAM and 2 to 4 vCPU.

Dockerize nginx, MySQL and WordPress

I’ve been playing with this already. I’ve picked up my own combo of favorite/useful WordPress plugins, so I’ll probably share a generic set of Dockerfiles and then leave it up to anyone if they want to use them to customize your own WordPress install in the container.

Configure a local dev/test environment Docker setup vs production environment Docker setup on my VPS

This makes a lot of sense and is a benefit of using containers. This will allow me to test my config locally, and then push to my production node. I’ve been looking at using Rancher to help with this, but still got lots to learn.

More updates to come as my project progresses.

Returning custom HTTP headers from nginx

Using curl -v against your site you can easily check the headers being returned from your nginx server. To see only the HTTP exchange without the actual HTML content you can send the output to /dev/null:

curl -v https://yoursite.com -o /dev/null

By default my site is returning:

< Server: nginx
< Date: Sat, 29 Jul 2017 20:12:10 GMT
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Link: <https://www.kevinhooke.com/wp-json/>; rel="https://api.w.org/"
< Link: <https://wp.me/91fMZ>; rel=shortlink

To add additional headers, such as XSS prevention headers:

X-XSS-Protection: 1; mode=block

Edit your nginx.conf and add to your server { } block:

add_header X-XSS-Protection "1; mode=block";

To hide X-Powered-By headers if you are using fastcgi (see here), add:

fastcgi_hide_header X-Powered-By;