AmazonDynamoDBException: Consistent reads are not supported on global secondary indexes

If you try to execute a DynamoDB query against a Global Secondary Index, by default you’ll get this error:

AmazonDynamoDBException: Consistent reads are not supported on global secondary indexes (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException

To avoid this, per answers on this question, you need to call:

.withConsistentRead(false);

on your query expression.

Although the docs here suggest queries are eventually consistent by default, apparently with the DynamoDB mapping api you still need to explicitly set the consisten read = false parameter.

Building bots on Twitter with AWS Lambdas (and other stuff)

I’ve built a few different bots on Twitter and written several articles describing how I built them. Some of these were a few months back – once they’re up and running it’s easy to forget they’re up and running (thanks to the free tier on AWS Lambda which means you can run scheduled Tweets well within the free tier limits). This is a summary of the bots I’ve developed so far.

Looking at where I got started, my first bot was to build an integration between Amateur Radio 2m Packet, retweeting packets received locally to Twitter. This was my first experience working with the Twitter REST apis and the OAUTH authentication, so I lot of what I learned here I reapplied to the following bots too:

For my next project, I was inspired by articles by researcher Janelle Shane who has been training ML models to produce some hilarious results, such as weird recipes, college course names and many others. I was curious what content a ML model would generate if I extracted all of my past 4000+ Tweets from Twitter and trained a model with the content. I had many questions, such as would the content be similar in style, and is 4000 Tweets enough text to train a model? You can follow my progress in these posts:

This then led to repeating the experiment with over 10 years of my blog articles and posts collected here, which you can follow in these posts:

Next, what would it take to train my model in the cloud using AWS Sagemaker, and run using AWS Lambdas?

You can follow this bot on Twitter here: @kevinhookebot

I had fun developing @kevinhookebot – it evolved over time to support a few features, not just to retweet content from the trained ML model. Additional features added:

  • an additional Lambda that consumes the Twitter API ‘mentions’ timeline and replies with one of a number of canned responses (not generated, they’re just hard coded phrases). If you reply to any of it’s tweets or Tweet @ the bot it will reply to you every 5 minutes when it sees a new tweet in the mentions timeline
  • another Lambda that responds to @ mentions to the bot as if it is a text-base adventure game. Tweet ‘@kevinhookebot go north’ (or east/west/south) and the bot will respond with some generated text in the style of an adventure game. There’s no actual game to play and it doesn’t track your state, but each response is generated using @GalaxyKate ‘s Tracery library to generate the text using a simple grammar that defines the structure of each reply.

After having fun with the adventure text reply generator, I also used the Tracey library for another AWS Lambda bot that generates product/project names and tweets every 6 hours. I think it’s rather amusing, you can check it out here: @ProductNameBot : 

@ProductNameBot

My most recent creation I upped the ante slightly and wondered what it would take to develop a Twitter bot that playeda card game. This introduced some interesting problems that I hadn’t thought about yet, like how to track the game state for each player. I captured the development in these posts here:

I have some other ideas for something I might put together soon. Stay posted for more details 🙂

Building a Card Playing Twitter Bot: storing and retrieving game state to/from AWS DynamoDB

I recently built a Twitter Bot that plays Blackjack. Here’s my previous posts so far if you are catching up:

Since interaction with the Bot is via Twitter, there will be an unknown length of time between when a player choses to reply and interact with the bot which could be seconds, minutes or days. We therefore need to store the gameplay state for each player, retrieve it on the next interaction from the user and store it again after the response from the bot while we wait for the next player interaction.

The bot can support any number of players in parallel. For each bot/player interaction we need to store the following:

  • the player’s twitter handle
  • current cards remaining in the deck
  • cards in the player’s hand
  • cards in the bot’s hand
  • who’s turn is it next

In a ‘traditional’ data model we could easily model this with some relational tables, but with DynamoDB as a key/value / document based datastore, since we only need to store one interaction state per user, we can store this whole data structure in a single row keyed by the user’s Twitter handle.

DynamoDB’s supported types of scalar values (number, string), and set types (collections of scalar types) allow us to store everything we need in the game state (I did consider the document type, persisting JSON documents), but for retrieving game state values in an easy to use format this didn’t appear as useful and straightforward as scalar types and sets).

Browsing the table in DynamoDB using the AWS console, here’s what the schema currently looks like for a completed game:

AWS DynamoDB offers 3 different APIs for interacting with tables and your data, described in the docs here: low level, document, high level object mapper. With the AWS Java SDK APIs, using Java POJO classes to represent your data together with the DynamoDB Object Mapper APIs is probably the simplest of the 3 approaches, and will feel familiar if you’ve used JPA, Hibernate or other object/relational type mappers before.

Similar to JPA, the AWS DynamoDB APIs provide a number of annotations to map Java Pojos to your DynamoDB tables and columns. Here’s the Pojo class that represents the game state:

[code language=”Java”]
@DynamoDBTable(tableName = "twitterbot-blackjack")
public class PlayerGameState {

private String twitterhandle;
private Hand playerHand;
private Hand botHand;
private Deck deck;

}
[/code]

twitterhandle is the key, the annotation for the key column looks like this:

[code language=”Java”]

@DynamoDBHashKey
public String getTwitterhandle() {
return this.twitterhandle;
}

[/code]

deck, playerHand and botHand are all collections of Card. As a ‘sub-document’ type used by each of these other collections of Cards, the type is annotated with @DynamoDBDocument (instead of @DynamoDBTable):

[code language=”Java”]

@DynamoDBDocument
public class Card {

private Suit suit;
private CardName name;
private int pointsValue;


}
[/code]

DynamoDB supports maps of scalar values, so these fit well for representing a deck of cards, and the player’s hands of cards. If a single Card is a map of values, a collection of Cards is a map of Cards, so a map of maps. To map these more complex structures, a DynamoDB Type Converter is needed to tell the DynamoDB api how to map the structure to the table and back:

[code language=”Java”]
@DynamoDBDocument
public class Hand {

private List<Card> cards = new ArrayList<>();

@DynamoDBTypeConverted(converter = ListOfCardsToMapOfMapsConverter.class)
public List<Card> getCards(){
return this.cards;
}

}
[/code]

Next up I’ll describe how these Type Converters are used in more detail, and we’ll look at storing an retrieving from a DynamoDB table.