AWS Lambda errors with Java Lambdas and Serverless framework (502 Malformed Lambda proxy response)

Using a basic Servlerless framework event config like this:

functions:
  hello:
    handler: kh.javalambda.HelloLambda::myHandler
    events:
      - http:
        path: hello
        method: get 

…will create an API Gateway config with the Lambda Proxy feature enabled.

This sends the request as a JSON object to the Lambda. This includes HTTP headers, queryStringParameters, pathParameters and the request body. The Lambda is expected to have an appropriate parameter type for the incoming JSON payload otherwise you’ll get a Jackson parsing error like this:

An error occurred during JSON parsing: java.lang.RuntimeException
 java.lang.RuntimeException: An error occurred during JSON parsing
 Caused by: java.io.UncheckedIOException: com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token
  at [Source: lambdainternal.util.NativeMemoryAsInputStream@1bce4f0a; line: 1, column: 1]
 Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token
  at [Source: lambdainternal.util.NativeMemoryAsInputStream@1bce4f0a; line: 1, column: 1

Use a Map<String, Object> as shown in the docs and the incoming request JSON will get passed in as this map.

Even if you are calling the Lambda with a GET request and not intending to POST or PUT a request body, you still need to have the Map<String, Object> parameter to receive the JSON event containing all the headers and params passed from API gateway.

Return Type

If you return a regular String or anything other than the expected response payload, from your Lambda will cause API Gateway to fail parsing the expected JSON response:

Wed Jan 13 07:34:32 UTC 2021 : Endpoint response body before transformations: "HelloLambda: hello null" Wed Jan 13 07:34:32 UTC 2021 : Execution failed due to configuration error: Malformed Lambda proxy response Wed Jan 13 07:34:32 UTC 2021 : Method completed with status: 502

With the API Gateway Lambda Proxy feature enabled, API Gateway expects response payloads from your Lambda to match the format shown here in the docs. Serverless framework generates a class called APIGatewayResponse that matches the expected format. Either create your own POJO that matches the expected return format or use the generated APIGatewayResponse class.

The generated sample Handler class shows this in use:

return ApiGatewayResponse.builder()
  .setStatusCode(200)
  .setObjectBody(responseBody)
  .setHeaders(Collections.singletonMap("X-Powered-By", "AWS Lambda & serverless"))
  .build(); 

Creating a new AWS Lambda project with Serverless

Assuming the Serverless cli is already installed (here), init a new project with ‘serverless’ and answer the following questions:

% serverless       
 Serverless: No project detected. Do you want to create a new one? Yes
 Serverless: What do you want to make? AWS Node.js
 Serverless: What do you want to call this project? lambda-example
 Project successfully created in 'lambda-example' folder.

To deploy, run ‘serverless deploy’

Interrelated personal side projects can be a goldmine of learning opportunities

Some people struggle to find ideas for working on side/personal software development projects for learning (and fun!). I’ve kept a running list of ideas in an online notebook and if I’m struggling for an idea for a new project I refer back to this list.

Over the past couple of years I stumbled across a series of inter-related projects that’s been a goldmine of learning opportunities, from frontend to backend to serverless.

This series of projects started with a question “Can you solve a problem without understanding the problem?”. tl;dr? No. The problem for this activity was solving Sudoku puzzles, and you can read more about this here:

This lead to the next step, researching more about exact cover problems, and how they can be solved with well established algorithms:

After building an implementation of Donald Knuth’s Algorithm X, I packaged it up as an AWS Lambda, and then built a React frontend for it:

I have one other project in progress for the frontend, replacing Flux with Redux. I’m still working on that one.

After building my solver and the frontend, I starting thinking about how to generate new puzzles. This is in progress here: https://github.com/kevinhooke/sudoku-generator

If you’re generating new puzzles you also need a way to grade the difficulty of puzzles. The unusual thing about this is there’s no established algorithm for grading the complexity of Sudoku puzzles, they’re typically graded by applying human solving techniques, so this led to this: https://github.com/kevinhooke/sudoku-human-grader

This string of related projects has kept me busy of the past couple of years. Not every idea will lead to a series of related projects like this, but if you can find an idea that does, it will keep you busy with plenty of problems to solve and many learning opportunities.

node.js, node-oracledb and Oracle Instant Client

To access an Oracle DB from an AWS Lambda function developed with node.js, you need to package you Lambda with shared libraries from Oracle’s Instant Client. The install instructions are here ( http://oracle.github.io/node-oracledb/INSTALL.html#quickstart ) but the only part that is really needed is the download location (since there’s no specific instructions for bundling the libs with an AWS Lambda): https://www.oracle.com/database/technologies/instant-client/linux-x86-64-downloads.html

Not all the Oracle Instant Client files are needed. From this older npm module to automate the packaging of the required libraries, I used this same list of required libraries:

libclntshcore.so.19.1
libclntsh.so.19.1
libmql1.so
libipc1.so
libnnz19.so
libons.so (not packaged in current Instant Client)
libociicus.so
libaio.so (from separate download - see next step)

libaio – if you’re on a Linux platform you can ‘apt-get install libaio’ or similar, but building my Lambda on a Mac I had to manually download the package and extract just the .so file from here (download the Arch Linux x64 package): https://pkgs.org/download/libaio

Put these in a /lib dir and zip up the folder and files. Use this to create a Lambda Layer.

For the Lambda itself install the node.js module for the api:

npm install –save node-oracledb

For examples in api usage, see the examples here: https://github.com/oracle/node-oracledb/tree/master/examples