Creating a Serverless framework project with Java Lambdas

The Serverless.com cli gives 2 Lambda project type options for new projects – Node,js and Python:

% serverless  
 Serverless: No project detected. Do you want to create a new one? Yes
 Serverless: What do you want to make? 
   AWS Node.js 
   AWS Python 
 ❯ Other 

If you select Other, it prompts you to create a project using a template:

Run “serverless create --help” to view available templates and create a new project from one of those templates.

The ‘create –help’ option tells you to run with the –template option and provides a long list of supported project types. Since I’m using Maven with Java, I’ll use the aws-java-maven option:

serverless create --template 

Since I already had a Maven pom.xml in place as a starting point for my Lambdas in this test project, the serverless cli warns that it won’t overwrite the existing file. I’m not familiar with what additional dependencies the aws-maven-template will add, so I renamed my pom.xml and reran the ‘serverless create’ cli and generated a new pom.xml.

Looking in the new file, there’s a similar and expected use of the Maven Shade plugin to bundle a ‘fat jar’ and other dependencies for Log4J and the addition of Jackson for json parsing.

There’s also a couple of extra Classes generated too that I wasn’t expecting, but they match up with the example code in the serverless docs (article here), so there’s a ApiGatewayResponse class that I wasn’t familiar with (from building AWS Lambdas with Java by hand and not using the API Gateway Lambda Proxy feature).

As a test, I looked into creating a couple of Java Lambdas not using the generated Classes just to confirm that there’s nothing Serverless framework specific that needs to be used. As it turns out, the default usage of the APIGateway Lambda Proxy feature the Lambda runtime is is expecting to map a json payload into the handler parameters and similarly for the response payload. For testing I just wanted to pass a couple of String request params on a GET request. So for my first test I got the following exception:

An error occurred during JSON parsing: java.lang.RuntimeExceptionjava.lang.RuntimeException: An error occurred during JSON parsingCaused by: java.io.UncheckedIOException: com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token at [Source: lambdainternal.util.NativeMemoryAsInputStream@4cf777e8; line: 1, column: 78] (through reference chain: java.util.LinkedHashMap["headers"])Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token at [Source: lambdainternal.util.NativeMemoryAsInputStream@4cf777e8; line: 1, column: 78] (through reference chain: java.util.LinkedHashMap["headers"])    at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)  

At this point I got distracted with a wide range of Java Lambda together with the API Gateway Lambda Proxy specific issues which I covered in a separate post here.

Long story short, the Serverless config for a Java Lambda enables the API Gateway Lambda Proxy feature by default, which means your Lambda impl needs to have a POJO class for it’s return type that matches exactly what API Gateway expects, so the Lambda to API Gateway Proxy integration can map the return value to the expected JSON structure. You can build this yourself to match what is described in the docs (link above) or just use the provided class generated by the aws-java-lambda template. The generated class ApiGatewayResponse is exactly what you need, so rather than reinventing the wheel I changed to use this generated class as the return value from my Java Lambda handler and now it works as expected.

My handler now looks like this:

public class MyHandler implements RequestHandler, ApiGatewayResponse> {

    @Override public ApiGatewayResponse handleRequest(Map<String, Object> input, 
        Context context) {
    }
}

Note that in order to receive parameters from incoming requests via API Gateway proxy, the first parameter needs to be a Map<String, Object>.

This is the first time I’ve used API Gateway Lambda Proxy with Java Lambdas. Previously the JavaLambdas I’ve built took advantage of API Gateway mapping any parameters to your Lambda automatically using Jackson to a POJO parameter on your Handler method, and even handing a POJO return type serializing that to a JSON response for you. I’ll come back and do some comparisons between these two approaches later.

To deploy your Java Lambda using serverless it’s the same as with Node.js Lambdas or any other supported runtime:

serverless deploy

To test calling your Java Lamdba function locally as if it’s deployed to AWS, use

serverless invoke local --function functionName

where functionName is what to defined your handler as in your serverless.yml.

By default the generated ApGatewayResponse class doesn’t have a toString() so you’ll see the response to your local test print something like:

com.serverless.ApiGatewayResponse@9301672

but you can add a toString() to help with testing locally (this is mentioned in the docs here).

The servless.com framework saves a lot of time in automating the deployment and configuration of your Lambdas and is well worth a look.

AWS Lambda errors with Java Lambdas and Serverless framework

Using a basic Servlerless framework event config like this:

functions:
  hello:
    handler: kh.javalambda.HelloLambda::myHandler
    events:
      - http:
        path: hello
        method: get 

…will create an API Gateway config with the Lambda Proxy feature enabled.

This sends the request as a JSON object to the Lambda. This includes HTTP headers, queryStringParameters, pathParameters and the request body. The Lambda is expected to have an appropriate parameter type for the incoming JSON payload otherwise you’ll get a Jackson parsing error like this:

An error occurred during JSON parsing: java.lang.RuntimeException
 java.lang.RuntimeException: An error occurred during JSON parsing
 Caused by: java.io.UncheckedIOException: com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token
  at [Source: lambdainternal.util.NativeMemoryAsInputStream@1bce4f0a; line: 1, column: 1]
 Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token
  at [Source: lambdainternal.util.NativeMemoryAsInputStream@1bce4f0a; line: 1, column: 1

Use a Map<String, Object> as shown in the docs and the incoming request JSON will get passed in as this map.

Even if you are calling the Lambda with a GET request and not intending to POST or PUT a request body, you still need to have the Map<String, Object> parameter to receive the JSON event containing all the headers and params passed from API gateway.

Return Type

If you return a regular String or anything other than the expected response payload, from your Lambda will cause API Gateway to fail parsing the expected JSON response:

Wed Jan 13 07:34:32 UTC 2021 : Endpoint response body before transformations: "HelloLambda: hello null" Wed Jan 13 07:34:32 UTC 2021 : Execution failed due to configuration error: Malformed Lambda proxy response Wed Jan 13 07:34:32 UTC 2021 : Method completed with status: 502

With the API Gateway Lambda Proxy feature enabled, API Gateway expects response payloads from your Lambda to match the format shown here in the docs. Serverless framework generates a class called APIGatewayResponse that matches the expected format. Either create your own POJO that matches the expected return format or use the generated APIGatewayResponse class.

The generated sample Handler class shows this in use:

return ApiGatewayResponse.builder()
  .setStatusCode(200)
  .setObjectBody(responseBody)
  .setHeaders(Collections.singletonMap("X-Powered-By", "AWS Lambda & serverless"))
  .build(); 

Creating a new AWS Lambda project with Serverless

Assuming the Serverless cli is already installed (here), init a new project with ‘serverless’ and answer the following questions:

% serverless       
 Serverless: No project detected. Do you want to create a new one? Yes
 Serverless: What do you want to make? AWS Node.js
 Serverless: What do you want to call this project? lambda-example
 Project successfully created in 'lambda-example' folder.

To deploy, run ‘serverless deploy’

Using Serverless framework and AWS sts:assume-role to cross deploy to different AWS accounts

In order to assume a role in another account, the owning account needs to grant a ‘trust relationship’ to those allowed to assume the role. This can be done by referencing an IAM username or role for those in the other account that are allowed to assume this role.

You can do this in the Console using the Trust Relationship tab:

A Policy to grant access to to a specific IAM user looks like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::ACCOUNT-ID:user/USER-ID"
      },
    "Action": "sts:AssumeRole"
    }
  ]
}

To assume this role, use the ‘aws sts assume-role’ cli command:

aws sts assume-role --role-arn arn:aws:iam::ACCOUNT-ID:role/ROLE-NAME --role-session-name SESSION-NAME

If this is successful, you’ll see a response that grants temporary values for the following AWS credentials that can be used after this point:

  • AccessKeyId
  • SecretAccessKey
  • SessionToken

The returned values can be used to set env vars to use with the CLI and other AWS SDK apps:

  • export AWS_ACCESS_KEY_ID=
  • export AWS_SECRET_ACCESS_KEY=
  • export AWS_SESSION_TOKEN=
  • export AWS_DEFAULT_REGION=

For Servlerless to deploy into another account, if you attempt a Serverless deploy at this point, you’ll see errors like:

User: arn:aws:sts::ACCOUNT-ID:assumed-role/ServerlessLambdaDeployRole/lambdadeploy is not authorized to perform: cloudformation:CreateStack on resource: arn:aws:cloudformation:us-east-1:TARGET-ACCOUNT-ID:stack/deploy-demo/*

In this case cloudformation:CreateStack is missing from the assumed role. If you incrementally attempt to find what additional permissions you’ll need to deploy, you’ll also need to add:

  • cloudformation:DescribeStackEvents
  • cloudformation:DescribeStackResource
  • cloudformation:ValidateTemplate
  • cloudformation:UpdateStack
  • cloudformation:DeleteStack
  • apigateway:POST
  • iam:CreateRole
  • iam:PutRolePolicy

ValidateTemplate appears to throw an error unless the Resource is for a wildcard of ‘*’ and not anything more specific, otherwise you’ll see this error:

Error: The CloudFormation template is invalid: User: arn:aws:sts::ACOUNT-ID:assumed-role/ServerlessLambdaDeployRole/lambdadeploy is not authorized to perform: cloudformation:ValidateTemplate

To grant permissions for ValidateTemplate specify a Resource of “*”

{
"Sid": "CreateCloudFormationStackValidate",
"Effect": "Allow",
"Action": [
"cloudformation:ValidateTemplate"
],
"Resource": "*"
}

The STS temporary credentials will expire after 1 hour, so if you see this error:

An error occurred (ExpiredToken) when calling the AssumeRole operation: The security token included in the request is expired

then you’ll need to rerun the ‘aws sts assume-role’ command again. If you previously set the session token in AWS_SESSION_TOKEN, you’ll need to set it back to blank (along with AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) before you run the command again. When you get the refreshed values, remember to set the env vars with the updated values.

At this point, if you’ve run ‘aws sts assume-role’ and you’ve set the env vars for the returned temp credentials, you’ll be able to run a ‘serverless deploy’ and deploy into the other account where you’ve assumed this new role with the permissions to deploy. This should include permissions for creating a new Lambda and API Gateway, if you’re deploying anything else from your serverless config, you’ll need to add those permissions to the role you’re assuming.