rest – Kev's Development Toolbox

May 4, 2017

Building an Amateur Radio packet to Twitter bridge: Part 2 – Understanding Twitter’s API rate limits

Quick update on my project to build an Amateur Radio Packet to Twitter bridge (see here for the first post).

I’ve moved the node.js code to a Raspberry Pi 3. So far I have up and running:

OAUTH authentication with Twitter (covered in my first post)
Installed node.js on the Raspberry Pi
Parsing ax25 Packets read from the TNC-Pi using node.js ax25 library
MongoDB running on the Raspberry Pi
Partially implemented duplicate post detection / rate limiting

Twitter provides an incredibly useful REST based API to integrate with their service, but understandably your usage is ‘rate limited’ so you’re not retrieving vast amounts of data or flooding the service with spam Tweets.

To understand their rate limit approach, see this article: https://dev.twitter.com/rest/public/rate-limiting

For limits on POSTs, for example for creating new Tweets: https://support.twitter.com/articles/15364

Per above article, POSTs are limited 2,400 per day, or 100/hour. This means if you’re app could potentially use the POST apis more than 100 per hour, you need to build in some limiting logic in your app so you don’t exceed this threshold.

The rate limits for the /GET REST apis are here, although for my application I’m not relying heavily on GETs, the majority of the calls are POSTS:

https://dev.twitter.com/rest/public/rate-limits

Twitter’s api prevents you from posting duplicate Tweets (see here), although they don’t publish how much time needs to elapse before an identical post is no longer consider a duplicate (see here). This wasn’t something I was planning on handling, but given the repetitive nature of some packet transmissions, for example BEACON packets, and packets sent for ID, this has to be a necessary feature of my app, otherwise in the worst case it would be attempting to tweet a stations BEACON packet every minute continuously.

To handle this I’m storing each Tweet sent, with additional tracking stats, like time last Tweet sent. Using this time stamp I can then calculate the elapsed time between now and the time last sent to make sure enough time has elapsed before I attempt to Tweet the packet again.

This is still a work in progress and there’s still some tweeking to do on the duplicate post detection and rate limiting, but while testing you can see the Tweets sent from my Twitter account @KK6DCT_Packet.

Here’s an example of the packets currently being Tweeted:

Received station: N6YUV sending to: BEACON : N6YUV in Woodland, MBX is N6YUV-1

— KK6DCT-PacketTweets (@KK6DCT_Packet) May 4, 2017

February 16, 2017

Building a Spring Boot RestController to search Redis

I’ve just started taking a look at using Redis. I wondered what it would look like to build a simple REST interface with Spring Boot. Spring Data Redis makes this pretty simple.

First up, you need to configure a @Bean in your @SpringBootApplication class (full source is on github here):

@Bean
RedisTemplate<String, Object> redisTemplate() {
  RedisTemplate<String, Object> template = new RedisTemplate<String, Object>();
  template.setConnectionFactory(jedisConnectionFactory());

  // these are required to ensure keys and values are correctly serialized
  template.setKeySerializer(new StringRedisSerializer());
  template.setHashValueSerializer(new GenericToStringSerializer<Object>(Object.class));
  template.setValueSerializer(new GenericToStringSerializer<Object>(Object.class));
  return template;
}

This is connecting with default settings to a locally running Redis.

Wire up a RedisTemplate into a Controller like this:

@RestController
public class RedisRestController {

  @Autowired
  private RedisTemplate<String, Object> template;

  @GetMapping("singlekey/{key}")
  public RedisResult getSingleValue(@PathVariable("key") String key){
    String value = (String)this.template.opsForValue().get(key);
    RedisResult result = new RedisResult(key, value);
    return result;
  }
}

and this is a bare minimum to get started, but shows how easy it is to get stuff up and running with Spring Boot and other projects like Spring Data.

Send a GET to /singlekey/ with a key value and json for the result is returned.

February 13, 2017

Spring Boot RestController Error: “No converter found for return value of type”

Spring Boot RestControllers by default can return a Pojo class as the return result from a mapped request method, and it is converted into a Json result. I’ve run into this issue before though, and it’s not immediately obvious what’s wrong: ‘No converter found for return value of type’ :

java.lang.IllegalArgumentException: No converter found for return value 
of type: class kh.springboot.redis.domain.RedisResult
at org.springframework.web.servlet.mvc.method.annotation
.AbstractMessageConverterMethodProcessor
.writeWithMessageConverters(AbstractMessageConverterMethodProcessor.java:187) ~[spring-webmvc-4.3.6.RELEASE.jar:4.3.6.RELEASE]

My returned class from my @GetMapping method is just a simple Pojo:

public class RedisResult {

private String key;
private String value;

public RedisResult(String key, String value) {
  this.key = key;
  this.value = value;
  }
}

And my @RestController is a simple controller with a single @GetMapping (in this case I’m building a REST endpoint to query key values from Redis using Spring Data Redis RedisTemplate:

@RestController

public class RedisRestController {

  @Autowired
  @Qualifier("RedisTemplate")
  private RedisTemplate template;

  @GetMapping("singlekey/{key}")
  public RedisResult getSingleValue(@PathVariable("key") String key){
    String value = (String)this.template.opsForValue().get(key);
    RedisResult result = new RedisResult(key, value);
    return result;
  }
}

If you’ve found my post because you have this same issue, before you go down the rabbit hole adding additional Maven dependencies or additional annotations you think might be missing, the reason is usually that your Pojo class you are returning doesn’t have any public getter methods. For each property in your Pojo you want returned in your Json, make sure you have a public getter.

This is discussed in this StackOverflow post here.