Building a multi-container Spring Boot and MongoDB webapp with Docker 1.12.x

I’ve spent a bunch of time playing with apps in single containers up until now and reached the point where I started to think about questions like:

“ok, so how is this done if you have an application with multiple services, where you may need to scale individual services but not others?”

and

“how does an app in one container talk to another container?”

Composing applications from multiple containers and handling the scaling I think this is the sweetspot for Kubernetes, but I don’t feel ready to ramp up quite that much just yet. Besides, with Docker 1.12 adding ‘swarm mode’ and with docker-compose, it’s looking like Docker has most of the tools you need to build and scale a multi-container app out of the box without adding additional tools to the stack.

So here’s where I started:

  • Container 1: Spring Boot app with JAX-RS RESTful endpoints
  • Container 2: MongoDB database
  • Container 3: Data volume container for MongoDB

As it turns out, this is doesn’t involve too much additional work than just building Dockerfiles for the individual containers and then wiring them together with a docker-compose.yml file.

Starting with each container, here’s the pretty simple Dockerfiles for each:

Spring Boot container:

FROM java:openjdk-8-alpine
ADD SpringBootAddressBook-0.0.1-SNAPSHOT.jar /opt/SpringBootAddressBook-0.0.1-SNAPSHOT.jar
EXPOSE 8080
ENV MONGODB_DB_NAME addressbook
ENV MONGODB_DB_HOST mongo
ENV MONGODB_DB_PORT 27017
ENTRYPOINT ["java", "-jar", "/opt/SpringBootAddressBook-0.0.1-SNAPSHOT.jar"]

 

MongoDB container:
MongoDB can be run straight from the official dockerfiles on Docker Hub, using one container for the server and one for a data container – see the complete docker-compose file below.

Bringing it all together, here’s the Docker Compose file to orchestrate the containers together:

version: '2'
services:
    mongodata:
        image: mongo:3.2
        volumes:
        - /data/db
        entrypoint: /bin/bash
    mongo:
        image: mongo:3.2
        depends_on: 
            - mongodata
        volumes_from:
            - mongodata
        ports:
        #kh: only specify internal port, not external, so we can scale with docker-compose scale
            - "27017"
    addressbook:
        image: addressbook
        depends_on: 
            - mongo
        environment:
            - MONGODB_DB_NAME=addressbook
        ports:
            - 8080:8080
        links:
            - mongo

At this point, the group of containers can be brought up as a whole with:

docker-compose up

… and brought down with:

docker-compose down

You can individually scale any container with:

docker-compose scale containername=count

…. where count is the number of container instances to spin up.

 

So what if you want to add in a web frontend as a container too? Easy enough. Here’s an AngularJS frontend, served by nginx:

    web:
        image: docker-web-angularjs
        ports:
            - "80"

Now, if we spin up multiple containers for the REST backend and the nginx frontend as well, we need a load balancer as well, right? Also easy, just add in haproxy:

    lb:
        image: dockercloud/haproxy
        depends_on: 
            - addressbook
        environment:
            - STATS_PORT=1936
            - STATS_AUTH="admin:password"
        links:
            - addressbook
            - web
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
        ports:
            - 80:80
            - 1936:1936

I had noticed that the startup order of the containers was not always predictable, and sometimes the Spring Boot container would start before the MongoDB container was up. This can be fixed by adding the depends_on element. I’m not sure if I really needed to add all the dependencies that I did in order to force a very specific startup order, but this seems to work for me. The order I have in the complete docker-compose.yml is (from first to last):

  • mongodata (data container)
  • mongo
  • addressbook (REST backend)
  • web (AngularJS frontend)
  • haproxy

The complete source for the AddressBook backend is available in this project on github. The deploy-* folders contain the individual dockerfiles:

https://github.com/kevinhooke/SpringBootRESTAddressBook

 

JavaOne 2016: Oracle’s ‘refocusing’ of the EE8 proposal

Oracle’s planned Java EE ‘announcement‘ at JavaOne 2016 turned out to be (based on the announcement from the Sunday keynote and the Java EE 8 roadmap session by Linda Demichiel, EE spec lead for EE JSRs) a ‘refocusing’, a realignment to bring Java EE more in line with current trends, particularly to support deploying EE applications to the cloud and the development of microservices.

So what exactly is going to change? For the new EE8 spec proposal, with finalization now planned for 2017,  some previously proposed features for inclusion are now going to be dropped, namely:

  • MVC 1.0
  • JMS 2.1
  • EE Management apis 2.0

Given than MVC 1.0 was a new EE web framework as an alternative to JSF (not adding new technology but built on top of what we already have with a Reference Implementation already built out), this is perhaps the most surprising removal. Given the new focus of cloud and microservices however, this makes sense, as it doesn’t really add anything towards supporting the new goals.

Everything else in the original EE8 JSR 366 is going to remain in the new EE8 proposal.

New features to be added include a new feature for supporting Health Checks, and a new Configuration proposal to standardize external configuration of deployments – both of which will be useful to support deployments to the cloud.

Is this ‘refocusing’ enough? Is it going to be finalized soon enough? For an EE8 target of 2017, and EE9 of 2018, is it too little, too late?

Given that you can use current EE features today with frameworks like Spring Boot and Dropwizard to build and deploy microservices, is a standardized cloud/microservice focused EE platform still relevant?

Alternative runtimes like WildFly Swarm and IBM’s Liberty Profile are giving runtime flexibility and allow you to pick just the features your app needs at runtime (and are available today). Rather than an all-in-one EE runtime container, is a ‘pick n mix’ type approach more relevant to support the current popularity of microservices?

Notes from Raspberry Pi with Java 9 session – CON6210 – JavaOne 2016

A few interesting notes from the Java 9 Embedded on Raspberry Pi, session CON6210, presented by Stephen Chin.

Currently streaming live on YouTube: https://www.youtube.com/watch?v=v95y5CzvcZ4&feature=youtu.be

Full live stream schedule is here: https://community.oracle.com/docs/DOC-1005786

  • SE9 Embedded will be available in a ARM build for the Raspberry Pi
  • This ARM specific build will only be available for Pi 2 and 3 (not the original Pi 1 A/B and + models, as it’s been compiled for ARM Thumb-2 instructions, and these are only supported on Pi 2 ARM CPU and above
  • AWT, Swing and JavaFX will all be supported on SE9 Embedded on the Pi
  • Interesting side note about modular Jigsaw in SE9 – apparently the module file packaging and compression is moving away from zip based Jars, to another format that’s more efficient for class lookup/retrieval

Learning by doing

I’m often asked ‘how do I learn [insert something new here]?’, or ‘what approach do you use for learning something new?’. You can learn many ways, we all have different styles of learning so you should experiment to find out what works for you

In software development, learning by doing should never be ignored as a useful technique. It’s one thing to learn something at a high level by listening to a podcast, reading something online or in a book, but the next step if you really want to learn the ins and outs is to do it yourself.

Here’s an example of why this approach is valuable. You overhear a guy talking about how a machine is turned on: ‘to turn on the machine you press this button’. Some time later someone else asks you how to turn on the machines, and remembering the conversation earlier, ‘you press this button here’ – the other guy presses it, and nothing happens. Huh, what went wrong?

If you’d tried to turn on the machine yourself earlier you would have found out, probably by trial and error, that there’s a key and it needs to be turned to the ‘on’ position before you press the button. There’s no way you would have learned this just by the earlier conversation, but if you’d tried to turn on the machine yourself, if the key was in the ‘off’ position then the button wouldn’t have worked and so you would have started to investigate further, get the manual, or go and find the other guy and ask some questions.

This is rather contrived example to make a point, and how much detail you need or want to know about any given topic of course varies. However, in software development, I don’t think anyone would argue that you don’t learn to program by reading a book; you learn by doing, you learn by writing code. Go write some code, you might learn something.

Enabling Ubuntu Bash shell on Windows 10 with the Anniversary update

I already have the Anniversary Update on my Windows 10 install, and I needed to ssh into one of my Raspberry Pis. I normally do this from my Mac, but since I’m in Windows 10 working on something, I thought this would be a great opportunity to install the new Ubuntu Bash support and then I can ssh from there right? (I could just install Putty, but then I wouldn’t have a reason to check out the Ubuntu Bash support, right?)

First attempt following instructions here, the bash windows opened and closed quick, too quick to read whether there was an error or not.

Second attempt, from a Command Prompt, entering bash, I got this:

Ok, to enable Developer Mode, into Settings, and click the radio button under ‘Use Developer Features’:

Then I got this error:

Hmm. Not very helpful. Clicking on Learn More opened a browser to a help page, but it didn’t load – I’d reconfigured my ip address to a static address on a local network for testing, and I didn’t have internet access. 0x80004005 probably could be a little more helpful, but enabling DHCP to get an internet connection and then tried selecting the Developer Mode option again, it downloaded, said ‘Some features may not be available until your PC is rebooted’. Rebooted, ran bash from Command Prompt, and now it prompted to download the Ubuntu Bash support. Why it couldn’t have done this already is beyond me, and I’ve already rebooted once, but answering ‘y’ it starts downloading:

After it completes you create a unix user and then you’re all set, and it even has ssh included.

Capturing the KISS pseudo terminal /dev/pts/x value from Direwolf

If you startup Direwolf with a KISS pseudo terminal for connecting another KISS packet app through Direwolf, you probably also need to run kissattach and mkiss to attach to the pseudo terminal. You can do this manually, but after doing this a few times, I put together this script to grep for the returned /dev/pts/x and then execute the kissattach with this value:

iPhone 7 – where’s the innovation?

The iPhone 7 is probably going to be noticed more for what Apple has removed, rather than the faster processor and improved dual lens zoom camera, since this is the first iPhone without a headphone jack.

Apple is notable for removing features on their new products though – laptops and desktops (iMac) with no floppy drive, and then later, no optical drive (MacBook Air).

PCMagazine added this perfect statement:

People who want iPhones don’t want them because of the spec sheets. They want them because of iOS, third-party iOS apps, Apple’s service and support network, and the community effects of Apple-only systems like iMessage.

Apple has succeeded so well at creating desirable products, that if they announced an iPhone 8 today with no new or additional features over the just announced iPhone 7, people would still buy it, just because it’s “one better”.

Winlink amateur radio email via paclink-unix on the Raspberry Pi

This is my second attempt to get a Winlink client (see here for a high level overview of Winlink) working on the Raspberry Pi. I first tried Pat /wl2k-go but it crashed (I created a ticket on github to followup), so for my second attempt I took a look at paclink-unix.

This site has a very detailed step by step install and config steps – follow exactly and pay attention to any errors 🙂 : http://bazaudi.com/plu/doku.php?id=plu:install_plu

A few additional notes:

  • wl2kax25 did not compile for me with the ax25 version that I previously had installed, possibly from apt-get from the default repos. Once I noticed this was missing I went back to the steps in the doc above and downloaded the ax25 packages from source, compiled, installed, rebuilt paclink and now I had the wl2kax25 app
  • Editing the /usr/local/etc/wl2k.conf file: the email= value is your local user email address on your local device, in this case on the Pi (e.g. for me, pi@localhost). This is used when wl2ktelnet/wl2kax25 retrieves incoming messages and it sends them to this user. If you see the wl2k app downloading messages but they’re not showing up in your inbox, check this.
  • Checking /var/log/mail.log is very useful to see what’s happening to your outbound and inbound messages!

The usage sequence is:

  • send outbound message with mail client, e.g. alpine
  • run wl2ktelnet to send over an internet connection if you have one
  • or, run wl2kax25 to send over your configured ax25 stack

The setup I got working is:

  • Raspberry Pi, with alpine (regular email client), direwolf (packet soundcard modem) and ax25 (to link paclink to direwolf)
  • Rigblaster Advantage USB soundcard, connected to an Icom 880

To send over vhf to my nearest Winlink gateway, I used:

  • wl2kax25 -a 1 -c KG6SJT-10 via KBERR

Where

-a 1 is port 1 defined in my /etc/ax25/axports

-c is the call of the Winlink gateway I’m connecting to, and I’m connecting via a packet digipeater, KBERR.

Raspberry Pi Winlink client over 2m VHF using Pat/wlk2-go + Direwolf + ax25 (not working yet)

Some rough notes on getting this combination working. This is my first attempt in getting the Pat / wlk2-go Winlink client working on a Raspberry Pi. I’m using:

  • Direwolf as a packet soundcard modem
  • ax25 as the connection between the Pat Winlink client and Direwolf
  • a Rigblaster Advantage, as a USB soundcard interface between the Raspberry Pi and a 2m transceiver (an Icom ID-880H)

I already have Direwolf and ax25 networking configured from using my Pi as a Packet radio client (search for my previous blog posts on Direwolf and you’ll find my notes for installing and configuring). Following the instructions in the Pat wiki here, I added a new ax25 interface as a new line to the end of my /etc/ax25/axports files:

wl2k KK6DCT 0 255 7 2m winlink

The wl2k alias for the new port I think is the default that Pat looks for to connect.

Note: I already have Direwolf configured with a 1200 port for VHF packet so I reused this as my first test, but if you configure a 300 port for HF and then connect the Rigblaster to an HF radio, I think the setup will be much the same (I haven’t tried this yet, but this will be my next test).

Next I installed golang on the pi with:

sudo apt-get install golang

and then installed Pat with:

go get github.com/la5nta/pat

go get failed initially with $GOPATH not set. So I created go-workspace in my home dir, and then did

export GOPATH="$HOME/goworkspace"

and added this export to my .bashrc so it will be set next time I log on too.

Now the go get started downloading to goworkspace, but it seemed to hang before it completed. Alternatively, you can download a prebuild .deb file from the Pat github releases page and install with (I used this approach instead):

sudo dpkg -i pat_0.1.5_linux_armhf.deb

pat configure – opens an editor for the config file. I added my callsign in mycall, my winlink password.

To add an alias in the configuration for a connection via ax25, I added a connection to KG6SJT-10, which is my closes Winlink gateway on 2m:

“connect_aliases”: {
“KG6SJT-10”: “ax25:///KG6SJT-10”,
“telnet”: “telnet://{mycall}:CMSTelnet@server.winlink.org:8772/wl2k”
}

pat http – starts the web interface

From the web gui at localhost:8080 you can select the Connect menu item, select the alias added in the config above, but this blew up Pat when I tried it. Time to log a ticket:

2016/09/07 00:20:53 Connecting to KG6SJT-10 (ax25)…
fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0x0 pc=0x76f3d774]

runtime stack:
runtime.throw(0x5d0948, 0x2a)
/opt/go/src/runtime/panic.go:547 +0x78
runtime.sigpanic()
/opt/go/src/runtime/sigpanic_unix.go:12 +0x44

goroutine 53 [syscall, locked to thread]:
runtime.cgocall(0x457ff0, 0x10c6b7e4, 0x0)
/opt/go/src/runtime/cgocall.go:123 +0x11c fp=0x10c6b7c4 sp=0x10c6b7ac
github.com/la5nta/pat/vendor/github.com/la5nta/wl2k-go/transport/ax25._Cfunc_ax25_aton_entry(0x0, 0x10ed00b0, 0x0)
??:0 +0x38 fp=0x10c6b7e0 sp=0x10c6b7c4
github.com/la5nta/pat/vendor/github.com/la5nta/wl2k-go/transport/ax25.(*ax25Addr).setPort(0x10ed00a0, 0x10b5a250, 0x4, 0x0, 0x0)
/home/martinhpedersen/go/src/github.com/la5nta/pat/vendor/github.com/la5nta/wl2k-go/transport/ax25/ax25_linux.go:361 +0x64 fp=0x10c6b7f0 sp=0x10c6b7e0
github.com/la5nta/pat/vendor/github.com/la5nta/wl2k-go/transport/ax25.DialAX25Timeout(0x10b5a250, 0x4, 0x10b5a158, 0x6, 0x10d107cd, 0x9, 0x7a358200, 0xa, 0xcb8e4, 0x0, …)
/home/martinhpedersen/go/src/github.com/la5nta/pat/vendor/github.com/la5nta/wl2k-go/transport/ax25/ax25_linux.go:136 +0x16c fp=0x10c6b984 sp=0x10c6b7f0
github.com/la5nta/pat/vendor/github.com/la5nta/wl2k-go/transport/ax25.Dialer.DialURL(0x7a358200, 0xa, 0x10d108d0, 0x0, 0x0, 0x0, 0x0)
/home/martinhpedersen/go/src/github.com/la5nta/pat/vendor/github.com/la5nta/wl2k-go/transport/ax25/ax25.go:114 +0x278 fp=0x10c6ba8c sp=0x10c6b984
github.com/la5nta/pat/vendor/github.com/la5nta/wl2k-go/transport/ax25.(*Dialer).DialURL(0x7c39f8, 0x10d108d0, 0x0, 0x0, 0x0, 0x0)
<autogenerated>:1 +0xb8 fp=0x10c6baac sp=0x10c6ba8c
github.com/la5nta/pat/vendor/github.com/la5nta/wl2k-go/transport.DialURL(0x10d108d0, 0x0, 0x0, 0x0, 0x0)
/home/martinhpedersen/go/src/github.com/la5nta/pat/vendor/github.com/la5nta/wl2k-go/transport/dial.go:29 +0xbc fp=0x10c6bad4 sp=0x10c6baac
main.Connect(0x10d107c5, 0x11, 0x0)
/home/martinhpedersen/go/src/github.com/la5nta/pat/connect.go:108 +0x7b4 fp=0x10c6bc5c sp=0x10c6bad4
main.ConnectHandler(0x76457918, 0x10c1b200, 0x10bf8230)
/home/martinhpedersen/go/src/github.com/la5nta/pat/http.go:357 +0x68 fp=0x10c6bca8 sp=0x10c6bc5c
net/http.HandlerFunc.ServeHTTP(0x6208b4, 0x76457918, 0x10c1b200, 0x10bf8230)
/opt/go/src/net/http/server.go:1618 +0x34 fp=0x10c6bcb8 sp=0x10c6bca8
github.com/la5nta/pat/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0x10a114a0, 0x76457918, 0x10c1b200, 0x10bf8230)
/home/martinhpedersen/go/src/github.com/la5nta/pat/vendor/github.com/gorilla/mux/mux.go:107 +0x250 fp=0x10c6bd44 sp=0x10c6bcb8
net/http.(*ServeMux).ServeHTTP(0x10a0ea80, 0x76457918, 0x10c1b200, 0x10bf8230)
/opt/go/src/net/http/server.go:1910 +0x164 fp=0x10c6bd70 sp=0x10c6bd44
net/http.serverHandler.ServeHTTP(0x10a75680, 0x76457918, 0x10c1b200, 0x10bf8230)
/opt/go/src/net/http/server.go:2081 +0x190 fp=0x10c6bda0 sp=0x10c6bd70
net/http.(*conn).serve(0x10b841c0)
/opt/go/src/net/http/server.go:1472 +0xee4 fp=0x10c6bfcc sp=0x10c6bda0
runtime.goexit()
/opt/go/src/runtime/asm_arm.s:990 +0x4 fp=0x10c6bfcc sp=0x10c6bfcc
created by net/http.(*Server).Serve
/opt/go/src/net/http/server.go:2137 +0x3bc