Creating an AWS Aurora Serverless database

I’m looking for a low cost managed db in the cloud for a small project, so thought I’d take a look at setting up an Aurora Serverless db, as depending on usage (and my usage will be very low) it looks like it’s definitely the cheapest of all AWS RDS options.

From the Console, from RDS, I pressed the ‘Create Database’ button:

If you select Aurora, the Serverless option is way down the page here:

I kept all the defaults, but changes the capacity to the smallest options:

After taking note of generated credentials and pressing the last ‘Create Database’ button, the dialog said it would take a couple of minutes to provision, and it sure did. I wasn’t timing it but it was at least 10 minutes before it was ready. This was probably the longest I’ve every waiting to provision anything on AWS.

Once it was ready, I tried to use the online query editor, but looks like there’s an additional step to create a user:

This option is under Network and Security:

After applying the change with the immediate option, I created a test table:

Inserted a row:

And then selected all rows:


Looks good so far!

Revisiting my spotviz.info webapp: visualizing WSJT-X FT8 spots over time – part 6: Redesigning to take advantage of the Cloud

Following on from Part 1 and subsequent posts, I now have the app deployed locally on WildFly 17, up and running, and also redeployed to a small 1 cpu 1 GB VPS: http://www.spotviz.info . At this point I’m starting to think about how I’m going to redesign the system to take advantage of the cloud.

Here are my re-design and deployment goals:

  • monthly runtime costs since this is a hobby project should be low. Less that $5 a month is my goal
  • take advantage of AWS services as much as possible, but only where use of those services still meet my monthly cost goal
  • if there are AWS free tier options that make sense to take advantage of, favor these services if they help keep costs down

Here’s a refresher on my diagram showing how the project was previously structured and deployed:

As of September 2019, the original app is now redeployed as a monolithic single .war again to WildFly 17, running on a single VPS. MongoDB is also running on the same VPS. The web app is up at: http://www.spotviz.info

There’s many options for how I could redesign and rebuild parts of this to take advantage of the cloud. Here’s the various parts that could either be redesigned, and/or split into separate deployments:

  • WSJT-X log file parser and uploader client app (the only part that probably won’t change, other than being updated to support the latest WSJT-X log file format)
  • Front end webapp: AngularJS static website assets
  • JAX-WS endpoint for uploading spots for processing
  • MDB for processing the upload queue
  • HamQTH api webservice client for looking up callsign info
  • MongoDB for storing parsed spots, callsigns, locations
  • Rest API used by AngularJS frontend app for querying spot data

Here’s a number of options that I’m going to investigate:

Option 1: redeploy the whole .war unchanged as previously deployed to OpenShift back in 2015, to a VM somewhere in the cloud. Cheapest options would be to a VPS. AWS LightSail VPS options are still not as a cheap as VPS deals you can get elsewhere (check LowEndBox for deals), and AWS EC2 instances running 24×7 are more expensive (still cheap, but not as cheap as VPS deals)

Update September 2019: COMPLETE: original app is now deployed and up and running

Option 2: Using AWS services: If I split the app into individual parts I can incrementally take on one or more of these options:

  • Route 53 for DNS (September 2019: COMPLETE!)
  • Serve AngularJS static content from AWS S3 (next easiest change) (December 2019: COMPLETE!)
  • AWS API Gateway for log file upload endpoint and RestAPIs for data lookups
  • AWS Lambdas for handling uploads and RestAPIs
  • Rely on scaling on demand of Lambdas for handling upload and parsing requests, removing need for the queue
  • Refactor data store from MongoDB to DynamoDB

Option 3: Other variations:

  • Replace use of WildFly queue with AWS SQS
  • Replace queue approach with a streams processing approach, either AWS Kinesis or AWS MSK

More updates coming later.

Moving GoDaddy DNS to AWS Route 53

Before changing the DNS settings on GoDaddy, set up Route 53 to manage DNS for your domain first, because you’ll need the AWS DNS server names when updating the GoDaddy DNS config.

In the AWS Console, go to Route 53, then Hosted Zones, and press the ‘Created Hosted Zone’ button:

Enter your domain name, select ‘Public Hosted Zone’ and press ‘Create’:

At this point Route 53 will have assigned a list of DNS nameservers for your domain – remember this list for when you update the GoDaddy config

Next, press ‘Create Record Set’, select Type A record and enter the name of your subdomain, e.g. www, and enter the IP address of your server that this name should resolve to:

Now let’s update the GoDaddy config. By default, here’s the DNS entries managed by GoDaddy when you register a domain with them:

First step, cancel the GoDaddy nameservers. Do this from the ‘Change’ button:

Change the type to Custom:

Then enter the dns server names from Route 53 and press ‘Save’. You should now be done. Assuming the setup on Route 53 has propagated, hit your domain name in a browser and hopefully you’re up and running.

Creating a new AWS Lambda using the AWS CLI

It’s pretty easy to set up and configure a new AWS Lambda with the AWS Console, but if you’re iterating on some changes and need to redeploy a few times, the AWS CLI makes it pretty easy.

To create a new Lambda, assuming you have a .zip packaged up and ready to go:

aws lambda create-function --function-name example-lambda --zip-file fileb://example-lambda.zip --handler index.handler --runtime nodejs8.10 --role arn:aws:iam::role-id-here:role/lambda-role