Website down for 24 hours: SSL certificate update failed – checking the contents of your certificate bundle

My SSL certificate for this site was about to expire this week, so I paid for an update for another year and then proceeded to upload my new certificate bundle to my server. Having been through this process a few times, I have a couple of posts describing the steps for configuring nginx with SSL certs here:

… and how to create a certificate bundle here:

I normally concatenate the root, intermediate and my site certificate manually before uploading using the steps in the post above. This time though I noticed the updated certifcate had a bundle download, so I downloaded this and uploaded straight to my site and then restarted…

Unfortunately, since I run nginx in a Docker container, on restarting the container it failed and then went into a restart loop. While constantly failing and restarting like this, it’s not possible (that I know of) to ‘docker exec -it bash’ into the container since it hasn’t completely started. In hindsight maybe ‘docker log’ would have told be what I needed to know, but I wanted to look at the /var/log/nginx/error.log inside the container to see what the issue was. I found a neat trick to do this which I’ll cover in another post.

In the meantime, I found the error in the nginx error.log was this:

2024/06/12 16:31:02 [emerg] 56#56: SSL_CTX_use_PrivateKey_file("/etc/nginx/ssl-certs/my-site.key") failed (SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch)

This seemed odd since I generated my CSR for the new certificate on the server and had the key for the request and new certificate. This post for this error luckily had suggestions to look at the contents of the bundle, using:

openssl x509 -noout -text -in yourcert.cert

And exactly as one of the answers suggested, the ‘Subject:’ field in the certificate was not for my domain, it was for the CA instead. The bundle that I downloaded after purchasing my new certificate contained the CA and the Intermediate certs but not the cert for my domain… I should have followed my own instructions for combining all three and including my own site certificate.

I created my new bundle by hand, uploaded to my server and now everything is back to normal with the new SSL certificate.

In hindsight I should have tested my updates on my test server before upload direct to my live server, but since moving house recently I no longer have the HP rackserver I had before, on which I used to run a test server that mirrored the config of my live site. Lesson learned, I need to set up a new test server…

AWS Lambda – querying and updating the Lambda runtime with the AWS CLI

In a previous post I used jq to parse the responses from the aws cli for Lambdas to search for or parse the results for specific property values. Querying Lambda properties can also be done via the aws cli itself without using jq (although using jq is significantly more powerful).

If you’ve received an email from AWS about runtimes that are reaching End of Life, the emails usually contain an example use of the cli showing how to query for specific runtimes, for example:

aws lambda list-functions --function-version ALL --region us-west-1 --output text --query "Functions[?Runtime=='nodejs14.x'].FunctionArn"

The long term solution for upgrading a runtime should be to redeploy and test with the latest runtime and then redeploy your production value after testing. It is possible for a quick fix to update the runtime of a deployed Lambda directly via the aws cli. Once you’ve identified which Lambdas need to update, use this command to update:

aws lambda update-function-configuration --function NAME --runtime RUNTIMENAME

New AWS Console with ‘favorites’

I just got prompted to switch to the new AWS Console and found this new feature that I’ve always though would be useful, and now here it it 🙂 You can now flag specific services as your ‘favorites’ and they appear in a ribbon bar across the top of your console and in a widget on the console page. Here’s my 3 ‘favorited’ services in the bar at the top of the console:

If you go to ‘Action’ then ‘Add widgets’ you can also add a favorites section and drag it to a location on your console page:

Here it is at the top of my console page:

Previously we had the ‘Recently visited’ section on the Console, but the items in the list would move around depending which you most recently used. Now with favorites you can pin the services you’re most often using so you can more easily access them.