Category: Tech Notes

Posted on

Installing and Configuring Atlassian Confluence with MySQL in Docker Containers

Atlassian Confluence is already available as a Docker Image from the Docker Hub but you still need to provide a database instance for a production setup. Let’s build a docker-compose file to create a container from this image together with a container running MySQL.

First,  per the docs on the Docker Hub page, create an external folder /data/confluence that will get mounted as a volume by the Container.

This is my first version to get this working (keep reading for refining this to include a JDBC driver)

[code]

version: ‘3’
services:
confluence:
image: atlassian/confluence-server
restart: always
volumes:
– /data/confluence:/var/atlassian/application-data/confluence
ports:
– 8090:8090
– 8091:8091
confl-mysql:
build: ./mysql
restart: always
environment:
– MYSQL_RANDOM_ROOT_PASSWORD=yes
– MYSQL_DATABASE=confluence
– MYSQL_USER=confluence
– MYSQL_PASSWORD=your-password
[/code]

After hitting your-ip:8090 for the first time, you can pick the ‘My own database’ option:

To connect to a MySQL db you need to drop a MySQL JDBC driver into /opt/atlassian/confluence/confluence/WEB-INF/lib so at this point we’ve got a couple of options. We could either copy the JDBC driver into the container (but since containers are ephemeral we’d lose this change if we started a new container from the image), or take a step back and rebuild the image including the driver:

The right thing to do would be to rebuild a custom image including the driver. So let’s do that.

Download the MySQL Connector driver from here.

Let’s commit it into our project and add a new Dockerfile to build a modified version of the official Confluence image, which is simply just these two lines:

[code]

FROM atlassian/confluence-server
COPY mysql-connector-java-5.1.46.jar /opt/atlassian/confluence/confluence/WEB-INF/lib

[/code]

Update the docker-compose file to build this new image instead of using the provided one from Docker Hub. Replace:

[code]

image: atlassian/confluence-server

[/code]

with

[code]

build: ./confl-mysql

[/code]

(or your corresponding name of your custom image containing the above Dockerfile)

Now when we startup this container and hit the app, the JDBC driver was recognized and we’re on to the next config page for our database connection params:

Entering our credentials and pressing Test, we’ve got an error about the default encoding:

To address this, the Confluence setup docs here describe editing the my.cnf file in MySQL, or alternatively I could pass params. The MySQL docs have a chapter on configuring and running MySQL in Docker, and this Q&A on Stackoverflow describes passing the optional params in a command section in your docker-compose file.

My first attempt was to add this:

[code]

confl-mysql:
build: ./mysql
restart: always
command: character-set-server=utf8 collation-server=utf8_bin
[/code]

but the syntax was not quite right yet, resulting in the container startup in a restart loop, and this error appearing in the container logs:

/usr/local/bin/docker-entrypoint.sh: line 202: exec: character-set-server=utf8: not found

Reading docs for the command option, the command in the docker-compose file needs to be the command to start the app in the container as well as the optional params. So now I’m here:

[code]

confl-mysql:
build: ./mysql
restart: always
command: [mysqld, –character-set-server=utf8 –collation-server=utf8_bin]
[/code]

Now we’re getting closer. Logs from my MySQL container and how showing:

ERROR: mysqld failed while attempting to check config

command was: "mysqld --character-set-server=utf8 --collation-server=utf8_bin --verbose --help"

mysqld: Character set 'utf8 --collation-server=utf8_bin' is not a compiled character set and is not specified in the '/usr/share/mysql/charsets/Index.xml' file

Some Googling made me realize each of the params is command separated,  so next update is:

[code]
confl-mysql:
build: ./mysql
restart: always
command: [mysqld, –character-set-server=utf8, –collation-server=utf8_bin]
[/code]

and now we’ve got both containers starting up. The list of params should be updated to add all the optional params listed in the Confluence MySQL set up docs, otherwise you’ll get an error for each missing param. The complete list is:

command: [mysqld, --character-set-server=utf8, --collation-server=utf8_bin, --default-storage-engine=INNODB, --max_allowed_packet=256M, --innodb_log_file_size=2GB, --transaction-isolation=READ-COMMITTED, --binlog_format=row]

… and my VM has run out of diskspace, so time to expand my disk. Back shortly.

Ok, back. Restarted and now we’re in business:

Complete config and now the containers are up!

Posted on

Changing a GitLab Runner from ‘Locked to a Project’ to Shared

I have a GitLab Runner assigned to a project that I’d like to share with another similar project. Currently it looks like this:

Pressing the small edit icon, I can see these options:

I want to reuse this same runner, so I unchecked the ‘Lock to current projects’ checkbox.

Now if I go to the CI/CD settings for my other project I can see it is available, so I click ‘enable for this project’

Now my Pending Job that was triggered after my first push to my repo has kicked in and is being deployed to my test Docker server. Cool.

Posted on

Building a Card Playing Twitter Bot: storing and retrieving game state to/from AWS DynamoDB

I recently built a Twitter Bot that plays Blackjack. Here’s my previous posts so far if you are catching up:

Since interaction with the Bot is via Twitter, there will be an unknown length of time between when a player choses to reply and interact with the bot which could be seconds, minutes or days. We therefore need to store the gameplay state for each player, retrieve it on the next interaction from the user and store it again after the response from the bot while we wait for the next player interaction.

The bot can support any number of players in parallel. For each bot/player interaction we need to store the following:

  • the player’s twitter handle
  • current cards remaining in the deck
  • cards in the player’s hand
  • cards in the bot’s hand
  • who’s turn is it next

In a ‘traditional’ data model we could easily model this with some relational tables, but with DynamoDB as a key/value / document based datastore, since we only need to store one interaction state per user, we can store this whole data structure in a single row keyed by the user’s Twitter handle.

DynamoDB’s supported types of scalar values (number, string), and set types (collections of scalar types) allow us to store everything we need in the game state (I did consider the document type, persisting JSON documents), but for retrieving game state values in an easy to use format this didn’t appear as useful and straightforward as scalar types and sets).

Browsing the table in DynamoDB using the AWS console, here’s what the schema currently looks like for a completed game:

AWS DynamoDB offers 3 different APIs for interacting with tables and your data, described in the docs here: low level, document, high level object mapper. With the AWS Java SDK APIs, using Java POJO classes to represent your data together with the DynamoDB Object Mapper APIs is probably the simplest of the 3 approaches, and will feel familiar if you’ve used JPA, Hibernate or other object/relational type mappers before.

Similar to JPA, the AWS DynamoDB APIs provide a number of annotations to map Java Pojos to your DynamoDB tables and columns. Here’s the Pojo class that represents the game state:

[code language=”Java”]
@DynamoDBTable(tableName = "twitterbot-blackjack")
public class PlayerGameState {

private String twitterhandle;
private Hand playerHand;
private Hand botHand;
private Deck deck;
…
}
[/code]

twitterhandle is the key, the annotation for the key column looks like this:

[code language=”Java”]

@DynamoDBHashKey
public String getTwitterhandle() {
return this.twitterhandle;
}

[/code]

deck, playerHand and botHand are all collections of Card. As a ‘sub-document’ type used by each of these other collections of Cards, the type is annotated with @DynamoDBDocument (instead of @DynamoDBTable):

[code language=”Java”]

@DynamoDBDocument
public class Card {

private Suit suit;
private CardName name;
private int pointsValue;

…
}
[/code]

DynamoDB supports maps of scalar values, so these fit well for representing a deck of cards, and the player’s hands of cards. If a single Card is a map of values, a collection of Cards is a map of Cards, so a map of maps. To map these more complex structures, a DynamoDB Type Converter is needed to tell the DynamoDB api how to map the structure to the table and back:

[code language=”Java”]
@DynamoDBDocument
public class Hand {

private List<Card> cards = new ArrayList<>();

@DynamoDBTypeConverted(converter = ListOfCardsToMapOfMapsConverter.class)
public List<Card> getCards(){
return this.cards;
}

…

}
[/code]

Next up I’ll describe how these Type Converters are used in more detail, and we’ll look at storing an retrieving from a DynamoDB table.

Posted on

AWS Lambda error: HTTP 502 ‘Execution failed due to configuration error: Malformed Lambda proxy response’

Calling an AWS Lambda via API Gateway with the Lambda Proxy Integration option, you might see an HTTP 502 response and this message:

“Execution failed due to configuration error: Malformed Lambda proxy response"

Wed May 30 05:00:41 UTC 2018 : Execution failed due to configuration error: Malformed Lambda proxy response
Wed May 30 05:00:41 UTC 2018 : Method completed with status: 502

This is a rather cryptic message, but  it’s saying is the response is not in the expected format.

Per this doc, the expected response should be in this format:

{
    "isBase64Encoded": true|false,
    "statusCode": httpStatusCode,
    "headers": { "headerName": "headerValue", ... },
    "body": "..."
}
Proudly powered by WordPress