GitLab CI Runner artifacts size error: “too large archive”

As part of putting together a GitLab CI pipeline to build a Python deployment for an AWS Lambda, I ran into an issue with the size of the build dir that I’m zipping up ready for deployment:

Uploading artifacts...
./build: found 2543 matching files 
ERROR: Uploading artifacts to coordinator... too large archive id=181 responseStatus=413 Request Entity Too Large status=413 Request Entity Too Large token=rtRUzgtp
FATAL: Too large

Hmm. Ok. A quick search found this post which says there’s a setting to increase the max build artifact size – it’s under /admin/application_settings, in the Continuous Integration setting – looks like the default is 100MB, so let’s bump that up and try again:

Setting up a new shared GitLab runner

Assuming you already have a runner installed (follow steps here).

Navigate to the [your-gitlab]/admin/runners page and scroll down to “How to setup a shared Runner for a new project” section – copy the server url and registration token for the next section.

On the machine where you want to the runner to execute, run

sudo gitlab-runner register

when prompted, enter the url and token.

For info on available executors see here.

Error invoking Python based AWS Lambda: /var/task/torch/_C.so: invalid ELF header

I’m trying to package a Python based Lambda to AWS but when my Python handler is called, I get this error:

Unable to import module 'helloworld-lambda': /var/task/torch/_C.so: invalid ELF header

Searching around it appears some Python modules are platform specific, so when running ‘pip install –target’ to install modules on MacOS before zipping up and deploying as a Lambda, it appears this installs platform specific libraries that are not compatible with the AWS Lambda runtime.

I’ll try packaging up my app on Linux and then try again.