serverless framework remove error: “An error occurred: xyzRole – Cannot delete entity, must detach all policies first.”

Attempting a ‘serverless remove’ on a deployed error, I got this unexpected error I haven’t seen before:

An error occurred: xyzRole - Cannot delete entity, must detach all policies first.

A quick Google found an issue with the same error and an explanation. There is another policy attached to this Lambda that was not added via the serverless.yml for this stack, so CloudFormation is refusing to delete it. This answer describes exactly what I had done to add the new policy:

I also had added XRay to this Lambda via the Console, and this added an additional managed policy to enable Xray. Checking the IAM Role, here’s the XRay related policy that was added:

I deleted the XRay policy, but at this point serverless has already removed most of the stack but left the IAM role in place, but won’t delete the stack itself, so a manual delete from the Console completed the cleanup.

AWS Lambda temp file storage

Lambdas by default do not have write access to the instance filestorage where they are running. They do my default have 512MB of ephemeral storage mounted at /tmp – this space is include in the Lambda runtime costs by default and can be increased (with additional cost) in 1MB increments upto 10GB.

More info here: https://docs.aws.amazon.com/lambda/latest/dg/configuration-function-common.html#configuration-ephemeral-storage

AWS Lambda Docker container runtime error: Runtime exited with error: exit status 127

While testing my Lambda using a Docker container, when the Lambda is invoked, I’m getting this runtime error:

Process 17(bootstrap) exited: Runtime exited with error: exit status 127

Process exit status code 127 errors are usually a file not found error. Going back through the Lambda logs, luckily there’s an error telling me I have an error in my shell script:

/var/task/test.sh: line 5: output: command not found

Quick and easy fix.