Retraining my Recurrent Neural Net with content from this blog

Recently I trained torch-rnn with all my previous tweets, and used it to generate tweets from an AWS Lambda, which results in some incomprehensible but somewhat recognizable content typical of my software development tweets like this:

and this:

In most cases the vocab it’s generating new content with has a high occurance of words I’d typically use, so computer, software, hardware, code are all pretty common in the output. Training the model with 2000+ tweets of 240 characters or less though I don’t think is a particular great sample of data, so I wondered what it would be like if I trained it with more data.

I have 2000+ articles on my blog here, so I ran a sql query to extract all the post text to a file (see here), and then fed this 4MB file into the training script. The script has been running on an Ubuntu VM on my rack server for almost 24 hours at this point, and it’s probably the most load I’ve had on my server (the 1 vCPU on the VM is maxed, but the server itself still has plenty of free vCPUs and RAM remaining, but this one vCPU is currently running 100%). It’s getting a little on the warm side in my office right now.

The torch-rnn script to train your model writes out a checkpoint file of the model in progress so far about once every hour, so it’s interesting to see how the generated content improves with every additional hour of training.

Here’s some examples starting with checkpoint 1, and then a few successive checkpoints as examples, running with temperature 0.7 (which gives good results after more training, but pretty wacky output earlier in the training):

Checkpoint 1, after about 2 hours:

a services the interease the result pecally was each service installing this up release have for a load have on vileent there at of althe Mork’ on it’s deforver, a some for

Checkpoint 5:

Store 4 minimal and Mavera FPC to speed and used that the original remeption of the Container and released and problem is any sudo looks most chated and Spring Setting the Started Java tagger

Checkpoint 10:

react for Java EE development and do it compended to the Java EE of code that yet this showing the desting common already back to be should announced to tracker with the problem and expenting

Checkpoint 15:

never that means that all performance developers of the just the hand of a microsch phone as not support with his all additional development though it’s better with the same by worker apache

Checkpoint 19:

The Java becomes are your server post configuring Manic Boot programming code in the PS3 lattled some time this is the last of the Docker direction is one it and a check the new features and a few new communities on the first is seen the destining

Getting pretty interesting at this point! Interesting that certain words appear pretty regularly in the generated output, although I don’t think I’ve included them in articles that often. PS2 and PS3 appear a lot, programming and computer are expected given the frequency in the majority of my articles, and there’s a lot of Java, Microsoft, Oracle, Docker and containers showing up.

I’m not sure how much longer the training is going to run for on a 4MB text file which I didn’t think was that large, but it’s been running for almost 24 hours at this point. I’ll let it run for another day and then see what the output looks like then.

If you start to see the tweets looking slightly more coherent over the next couple of days, the AWS Lambda is starting to use content generated from these new checkpoints on this new model, so it should be slightly more natural sounding hopefully, given the larger input file for training the new model.

Caching Spring Boot RESTController responses with Spring Cache and Redis

Spring Boot provides easy integration for caching responses using a number of cache providers. See the section in the docs here. Depending on what type of response you’re trying to cache however, there’s a range of issues you can run into.

If you’re interested in a working configuration then skip to the end, otherwise what follows is the steps I went through to try and get this working as I wanted. If you run into one of these error messages (which are surprisingly descriptive) then maybe you’ll find this post from a search and this will help you get to a solution).

I’m using Redis as a cache provider. To enable this I added the following to my Spring Boot application.properties:

spring.cache.type=redis
spring.redis.host=localhost
spring.redis.port=6379

Add this dependency to your pom.xml:

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-cache</artifactId>
</dependency>

And then add Redis support:

<dependency>
  <groupId>org.springframework.data</groupId>
  <artifactId>spring-data-redis</artifactId>
</dependency>

<dependency>
  <groupId>redis.clients</groupId>
  <artifactId>jedis</artifactId>
</dependency>

I’m using JPA with h2 for testing, so I’ve also added:

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>

and then for h2:

<dependency>
  <groupId>com.h2database</groupId>
  <artifactId>h2</artifactId>
</dependency>

To enable caching support in your Spring Boot app, add the @EnableCaching annotation:

@SpringBootApplication
@EnableCaching

To enable support for using a RedisTemplate directly (I think Spring Boot Cache when configured to use Redis as your cache provided uses this config too) add this @Bean to your @Configuration class:

@Bean
public RedisTemplate<String, String> redisTemplate(RedisConnectionFactory cf) {
  RedisTemplate<String, String> redisTemplate = new RedisTemplate<String, String>();
  redisTemplate.setConnectionFactory(cf);
  return redisTemplate;
}

… and configure the Jedis client library:

@Bean
public JedisConnectionFactory redisConnectionFactory() {
  JedisConnectionFactory redisConnectionFactory = new JedisConnectionFactory();

  // Defaults
  redisConnectionFactory.setHostName("127.0.0.1");
  redisConnectionFactory.setPort(6379);
  return redisConnectionFactory;
}

You also need to configure the RedisCacheManager as the cache provider:

@Bean
public CacheManager cacheManager(RedisTemplate redisTemplate) {
  RedisCacheManager cacheManager = new RedisCacheManager(redisTemplate);

  // Number of seconds before expiration. Defaults to unlimited (0)
  cacheManager.setDefaultExpiration(300);
  return cacheManager;
}

The first requirement for caching your responses is that the response needs to be Serializable, otherwise you’ll see this error in your response from an @RestController:

{
“status”: 500,
“error”: “Internal Server Error”,
“exception”: “org.springframework.data.redis.serializer.SerializationException”,
“message”: “Cannot serialize; nested exception is org.springframework.core.serializer.support.SerializationFailedException: Failed to serialize object using DefaultSerializer; nested exception is java.lang.IllegalArgumentException: DefaultSerializer requires a Serializable payload but received an object of type [org.springframework.http.ResponseEntity]”
}

ResponseEntity is the wrapper you typically use for your responses from a @RestController, and it’s not Serializable. From this question here, this suggests using a SpEL expression to tell the cache support to use the body of the response as a the key, and not the ResponseEntity itself (which makes sense):

@Cacheable(cacheNames = "addresses", key="#response?.body")

With this you now get:

Null key returned for cache operation (maybe you are using named params on classes without debug info?) Builder[public org.springframework.http.ResponseEntity kh.springcloud.service1.Service1RestController.getAddresses(java.lang.String) throws org.springframework.web.client.RestClientException] caches=[addresses] | key=’#response?.body’ | keyGenerator=” | cacheManager=” | cacheResolver=” | condition=” | unless=” | sync=’false'”

Thinking what I really need is the id of the address I’m returning as the key, I modified the SpEL to this:

#response?.body.id

but this is not really what I want either if I’m attempting to return a list of Addresses from GET /addresses?state=CA. What I really want as the key of the results to be cached is the State value that I’m querying on, so trying this approach:

#request.getParameter('state')

this says request is null.

Looking at the examples in this question, it looks like you can reference the method parameter names directly, so:

@Cacheable(cacheNames = "addresses", key="'state")

Now trying a request we get this:

“message”: “EL1008E: Property or field ‘state’ cannot be found on object of type ‘org.springframework.cache.interceptor.CacheExpressionRootObject’ – maybe not public?”,

Ok, how about key=”{#state}” – this gives same error as before.

“message”: “Expression [{ #state ] @0: EL1044E: Unexpectedly ran out of input”,

I’m really reaching for a solution at this point 🙂 What if we step back and simplify the api we’re trying to cache, and add a findById api and try to cache this by id:

@GetMapping("/addresses/{id}")
@Cacheable(cacheNames = "addresses", key="#id")
public ResponseEntity<Address> getAddressByd(@PathVariable("id") Long id)

Ok, with this simple endpoint, we’re not retrieving anything by id, but a quick sanity check (removing the cacheable) shows the responses are coming back without an id. So there’s your problem:

{
  "addr1": "test3",
  "city": "testcity2",
  "state": "CA",
  "zip": "95616"
}

Taking a look at my mapping, as a test I’m running against an in memory h2 db, and was mapping my id with AUTO:

@Id
@GeneratedValue(strategy=GenerationType.AUTO)

It seems this doesn’t generate any id at all for the h2 db, so let’s switch to an identity table (does h2 have sequences? I’m not sure, but feel like this is going too far down a rabbit hole)

@Id
@GeneratedValue(strategy=GenerationType.IDENTITY)

Now what do we get in responses – the same response. Trying /addresses/1 vs /addresses/2 I can see that I have a response with id=1, but not for id=2

Looking at the mappings in the entity, I don’t have a getter/setter for the id, so let’s add one and try again. Ok, now we’re good:

{
  "id": 1,
  "addr1": "test3",
  "city": "testcity2",
  "state": "CA",
  "zip": "95616"
}

Let’s put the @Cacheable back and try again.

This cache config:

@Cacheable(cacheNames = "addresses", key="#response?.body?.id")

Still gives:

"Null key returned for cache operation"

Let’s try id directly:

@Cacheable(cacheNames = "addresses", key="id")

Now we get:

“message”: “EL1008E: Property or field ‘id’ cannot be found on object of type ‘org.springframework.cache.interceptor.CacheExpressionRootObject’ – maybe not public?”,

key=”#id” is back with this error, so it seems like this is the right config, but we’re still struggling with the ResponseEntity not being Serializable:

“message”: “Cannot serialize; nested exception is org.springframework.core.serializer.support.SerializationFailedException: Failed to serialize object using DefaultSerializer; nested exception is java.lang.IllegalArgumentException: DefaultSerializer requires a Serializable payload but received an object of type [org.springframework.http.ResponseEntity]”,

This issue on the SpringBoot project is exactly what we’re seeing, but the OP was told to post the question to SO:

https://github.com/spring-projects/spring-boot/issues/5017

This might the same question, if not then it’s pretty much the same question, but the only answer so far says what we know so far: ResponseEntity is not Serializable, but we don’t know yet how to fix this. It seems we need a custom Serializer configured, but not sure how we do that yet.

This post suggests how to change/set the serializer and key serializer in use and suggests:

template.setDefaultSerializer(new GenericJackson2JsonRedisSerializer());
template.setKeySerializer(new StringRedisSerializer());
template.setHashKeySerializer(new GenericJackson2JsonRedisSerializer());
template.setValueSerializer(new GenericJackson2JsonRedisSerializer());

Trying with this in place, we’ve got a different error, and this seems an issue with the Long key, instead of a String key.

It doesn’t look like there is a LongRedisSerializer, but searching around I found this question which suggests to use the GenericRedisSerializer for other key types, so replacing StringRedisSerializer with this:

redisTemplate.setKeySerializer(new GenericToStringSerializer<Long>(Long.class));

Now we’re making some progress. Calling the get by id api the first time, the sql is executed and the json response is returned from the endpoint. Calling it a second time we now get a different error:

com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of org.springframework.http.ResponseEntity: no suitable constructor found, can not deserialize from Object value (missing default constructor or creator, or perhaps need to add/enable type information?)

Starting the redis-cli and doing a ‘keys *’ to see what if anything has been inserted, there’s a couple of keys, one is “1”.

Doing a ‘GET 1’ we get:

127.0.0.1:6379> get 1

“{\”@class\”:\”org.springframework.http.ResponseEntity\”,\”headers\”:{\”@class\”:\”org.springframework.http.HttpHeaders\”},\”body\”:{\”@class\”:\”kh.springcloud.service1.domain.Address\”,\”id\”:1,\”addr1\”:\”test3\”,\”city\”:\”testcity2\”,\”state\”:\”CA\”,\”zip\”:\”95616\”},\”statusCode\”:\”OK\”,\”statusCodeValue\”:200}”

127.0.0.1:6379>

We’ve got a cached response! So the problem now looks like deserializing the cached data back out to a ResponseEntity<Address>

What if we try @Cacheable on the JPA query instead of trying to work out how to deserialize the ResponseEntity?

First attempt:

@Cacheable(cacheNames = "addresses", key="#id")
public Address findOne(Long id);

This gives us an error about the key being null:

Null key returned for cache operation (maybe you are using named params on classes without debug info?) Builder[public abstract kh.springcloud.service1.domain.Address kh.springcloud.service1.repo.AddressRepository.findOne(java.lang.Long)

Ok. This post mentions an alternative numbered param reference to the params, using the format #p0. Let’s try:

@Cacheable(cacheNames = "addresses", key="#p0")

This works! Calling the api the first time we get the select statement against the db (seen from the hibernate debug log output), but subsequent times there’s no additional select against the db, the value is returned from cache.

There doesn’t seem to be much info about how to handle deserializing the result back out as a ResponseEntity, when I have some time I’ll see if I can work out how to do this, but in the meantime, caching the responses from JPA is good enough and avoids hitting the db every time, so this works for now.