Today I learned after spending a while trying to debug why a later job in my pipeline couldn’t see a file from a previous job, that GitLab does not preserve files on the filesystem between stages, and even jobs. I guess this makes sense as your pipeline is running against what it currently in your repo, and not untracked files that have been created by your pipeline.
If you are generating new files, from for example Ansible generating files from templates, if the files are generated in one job and then you expect to use them in a later job in the pipeline, you need to tell GitLab that the files are ‘artifacts’ to preserve them.
In the case of generated files, they will be untracked files in git. Tell GitLab to publish them as artifacts with the following config:
generate-nginx-config test2:
stage: generate-templates
environment: test2
script:
- cd iac/ansible
- ansible-playbook -i test2.yml nginx-playbook.yml
# keep the file generated from ansible template, which is now
# untracked, so it can be used in following jobs
artifacts:
untracked: true
paths:
- nginx/config/etc/nginx/sites-available
tags:
- docker-test
This is a job in my pipeline for generating my nginx config based on the environment I’m deploying to. Note the untracked: true which tells GitLab to preserve the untracked files as artifacts.