Website down for 24 hours: SSL certificate update failed – checking the contents of your certificate bundle

My SSL certificate for this site was about to expire this week, so I paid for an update for another year and then proceeded to upload my new certificate bundle to my server. Having been through this process a few times, I have a couple of posts describing the steps for configuring nginx with SSL certs here:

… and how to create a certificate bundle here:

I normally concatenate the root, intermediate and my site certificate manually before uploading using the steps in the post above. This time though I noticed the updated certifcate had a bundle download, so I downloaded this and uploaded straight to my site and then restarted…

Unfortunately, since I run nginx in a Docker container, on restarting the container it failed and then went into a restart loop. While constantly failing and restarting like this, it’s not possible (that I know of) to ‘docker exec -it bash’ into the container since it hasn’t completely started. In hindsight maybe ‘docker log’ would have told be what I needed to know, but I wanted to look at the /var/log/nginx/error.log inside the container to see what the issue was. I found a neat trick to do this which I’ll cover in another post.

In the meantime, I found the error in the nginx error.log was this:

2024/06/12 16:31:02 [emerg] 56#56: SSL_CTX_use_PrivateKey_file("/etc/nginx/ssl-certs/my-site.key") failed (SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch)

This seemed odd since I generated my CSR for the new certificate on the server and had the key for the request and new certificate. This post for this error luckily had suggestions to look at the contents of the bundle, using:

openssl x509 -noout -text -in yourcert.cert

And exactly as one of the answers suggested, the ‘Subject:’ field in the certificate was not for my domain, it was for the CA instead. The bundle that I downloaded after purchasing my new certificate contained the CA and the Intermediate certs but not the cert for my domain… I should have followed my own instructions for combining all three and including my own site certificate.

I created my new bundle by hand, uploaded to my server and now everything is back to normal with the new SSL certificate.

In hindsight I should have tested my updates on my test server before upload direct to my live server, but since moving house recently I no longer have the HP rackserver I had before, on which I used to run a test server that mirrored the config of my live site. Lesson learned, I need to set up a new test server…

Creating a single SSL certificate bundle from CA root and intermediate certificates

When you purchase an SSL certificate for a domain, e.g. to secure HTTPS usage with your web server, your certificate download may provide you several files like this:

youdomain.com.crt : this is your domain certificate

Files that look this are the root and intermediate certificates:

USERTrust_RSA_Certification_Authority.crt

Sectigo_RSA_Domain_Validation_Secure_Server_CA.crt

Depending how your SSL vendor supplies these files, the second two files may be provided already bundled in one file or two separate files.

To combine them together into one bundle file, use cat to concat the files:

cat yourdomain_com.crt USERTrustRSAAddTrustCA.crt AddTrustExternalCARoot.crt > domain_com.ca-bundle.crt

Open the file and check that each start and end comment for each cert are on their own line and not on the same line (see here).

http-server with self-signed certs and enabling support in Chrome

To test loading a site locally with http-server you need to generate self-signed certs:

openssl req -newkey rsa:2048 -new -nodes -x509 -days 365 -keyout key.pem -out cert.pem

Now start up http-server using the new cert:

http-server -S -C cert.pem -o .

If you attempt to browse https://localhost:8080 though, Chrome will block self-signed certs by default. To allow self-signed certs, enter in your Chrome address bar:

chrome://flags/#allow-insecure-localhost

And then enable the highlighted setting:

Let’s Encrypt certificate expired on older Macs

I have a number of older machines that I use on a regular basis, so I’m no stranger to the struggles of not being able to browse current websites on older machines with older browsers and the typical SSL/TLS support issues that you run into. I was surprised to see this error this week on my 2008 Mac Pro running Mac OS X 10.11 El Capitan and a latest version of Chrome:

Looking at the certificate for any site not loading it looks like the certificate has expired:

I’m not seeing this on my other later/current machines though, so clearly something on these older machines is no longer getting updates. Browsing around a few other sites and seeing the same issue on many sites so it was not just limited to a single site, so I realized something else was going on. Some Googling found this article:

Following the steps to download the updated certificate from LetsEncrypt and install it into Keychain did the job.