Articles, notes and random thoughts on Software Development and Technology
If you’re running a Maven phase/goal and need to skip the execution of tests (because they’re temporarily failing), add one of the following options:
mvn -DskipTests phasename
mvn -Dmaven.test.skip=true phasename
Normally you’d only continue with the build if the tests are passing, but sometimes you may have a valid reason to force the build to continue regardless.
These options are covered in the Maven docs here.
As part of migrating this blog to Docker containers to move to a different VPS provider (here, here and here), I found myself repeating a number of steps manually, which always a good indication that there’s an opportunity to automate some or all of those steps.
Each time I made a change in the configuration or changed the content to be deployed, I found myself rebuilding the Docker image and either running locally, pushing to my test server, and eventually pushing to my prod VPS and running there.
I’m using a locally running GitLab for my version control, so to use its build pipeline features was a natural next step. I talked about setting up a GitLab runner previously here – this is what performs the work for your pipeline.
You configure your pipeline with a .gitlab-ci.yml file in the root of your repo. I defined 2 stages, build and deploy:
stages: - build - deploy
For my build stage, I have a single task which is to build my images using my docker-compose.yml:
build: stage: build script: - docker-compose build tags: - docker-test
For my deploy steps, I defined one for deploying to my test server, and one for deploying to my production VPS. This is the deploy to my locally running Docker server. It changes DOCKER_HOST to point to my test server, and then uses the docker-compose.yml again to bring down the running containers, and bring up the new containers with the updated images:
deploy-containers: stage: deploy script: - export DOCKER_HOST=tcp://192.x.x.x:2375 - docker-compose down - docker-compose up -d tags: - docker-test
And one for my deploy to production. Note that this step is defined with ‘when: manual’ which tells GitLab the task is only run manually (when you click on the ‘>’ run icon):
prod-deploy-containers: stage: deploy script: - pwd && ls -l - ./docker-compose-vps-down.sh - ./docker-compose-vps-up.sh when: manual tags: - docker-prod
Here’s what the complete pipeline looks like in GitLab:
With this in place, now any changes committed to the repo result in a new image created and pushed to my test server automatically, and when I’ve completed testing the changes I can optionally deploy the changes to my prod VPS hosted server.
Everything in IT has pros and cons, and as they say, ymmv. One of the things that has always interested me in IT is how your point of view, the lens through which you perceive ‘how things are done’, is always (and perhaps obviously) influenced by your current company/project/client/organizational experience.
If for example your current experience is (and perhaps has always been) traditional waterfall-type development lifecycle, then the very idea that it’s possible to build/test/deploy new functionality to production in anything less than 6 month release cycles, let alone daily, probably sounds like an impossibility.
However, if this is your experience (and to be honest most of my industry experience has been using more traditional methods), there is a light bulb moment when you realize what it means to be able to deploy to production in very frequent, short development cycles.
The faster/quicker you can deploy new features to production, the faster you can learn about what works and what doesn’t. If you find problems sooner, you can respond and fix issues sooner. Fixing problems sooner means problems don’t get out of control and grow into much larger problems that become more expensive and more difficult to fix later. Getting features in front of your users sooner give them the opportunity to provide feedback quicker, and either confirm that this is what they were expecting, or whether changes need to be made to get closer to exactly what it is that they are looking for.
Continuous Delivery is not a myth, there’s many organizations out there already successfully using techniques to deploy to production daily (Infoq.com have a great collection of presentations on their site).