This is part 2 following on from my previous post, investigating how to take advantage of AWS SageMaker to train a model and use it to generate text for my Twitter bot, @kevinhookebot.
From the AWS SageMaker docs, in order to get the data in a supported format to use to train a model, it mentions “A script to convert data from tokenized text files to the protobuf format is included in the seq2seq example notebook”
Ok, so from the SageMaker Notebook I created in part 1, let’s start it up via the AWS console:
Once started, clicking the ‘Open’ link to open the Jupyter notebook, we can open the seq2seq example which is in the ‘SageMaker Examples’ section:
From looking at the steps in this example Notebook, it’s clear that this character2character algorithm is more focused on translating text from source to destination (such as translating text in one language to another, as shown in this example notebook).
Ok, so this isn’t what I was looking for so let’s change gears. My main objective is to be able to train a new model using AWS SageMaker service, and generate text from it. From what I understand so far, you have two options how you can use SageMaker. You can either use the AWS Console for SageMaker to create Training Jobs using the built in algorithms, or you can use a Juypter notebook and define the steps yourself using Python to retrieve your data source, prepare the data, and train a model.
At this point the easiest thing might be to look for another Recurrent Neural Net (RNN) to generate characters to replace the Lua Torch char-rnn approach I was previously running locally on an Ubuntu server. Doing some searching I found char-rnn.pytorch.
This is my first experience setting up a Juypter notebook, so at this point I’ve no idea if what I’ve doing is the right approach, but I’ve got something working.
On the righthand side of the notetbook I pressed the New button and selected a Python PyTorch notebook:
Next I added a step to clone the char-rnn.pytorch repo into my notebook:
Next I added a step to use the aws cli to copy my data file for training the model into my notebook:
Next, adding the config options to train a model using char-rnn.pytorch, I added a step to run the training, but it gave an error about some Python modules missing:
Adding an extra step to use pip to install the required modules:
The default number of epochs is 2,000 which takes a while to run, so decreasing this to something smaller with –n_epochs 100 we get a successful run, and calling the generate script, we have content!
I trained with an incredibly small file to get started, just 100 lines of text, for a very short time. So next steps I’m going to look at:
- training with the full WordPress export of all my posts for a longer training time
- training with a cleaned up export (remove URL links and other HTML markup)
- automate the text generation from the model to feed my AWS Lambda based bot
I’ll share another update on these enhancements in my next upcoming post.
