If you want to build efficient CI/CD pipelines in GitLab CI you’ll probably end up using own Runners. In this article, we will show you how to install and configure them for a Node project.
This is the first post from our series about building robust CI/CD pipelines for Node-based projects, wherein we will focus on how to prepare the infrastructure for pipeline execution.
In GitLab CI a Runner is a dedicated component that is responsible for executing jobs that are part of a given pipeline. So essentially, it’s a piece of code which is running on some VPS or VM and can build, test and deploy your code. As a part of a free plan, GitLab gives you Shared Runners for free – at least for first 2000 minutes per month, but as you may suspect they are not very powerful in terms of CPU and RAM (see here for detailed specification). If you want to reduce the time needed to execute CI/CD pipeline, wise option would be to spin up a few VPS in cloud providers of your choice (we use Digital Ocean and VPSDime) and use them instead of shared machines on GitLab.
Let’s see how to do that!
In order to install Runner on Linux machine, first add the GitLab package repository and then install it:
$ curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash $ sudo apt-get install gitlab-runner
Because we will be using Docker please ensure it is also installed.
When Runner is installed on a target machine it can be now registered. Preferably, register it as a group Runner so that it can pickup jobs from any project under that group (more about project groups can be found in the documentation).
Before starting the registration process, go to GitLab and open your group CI/CD settings page, expand Runners and go to Set up a group Runner manually section. Copy URL and token.
Now, on a target machine register the Runner by issuing the following command in terminal:
$ sudo gitlab-runner register
Before Runner can execute any job it must be first configured. To set up it’s parameters issue the following command in the terminal:
$ sudo nano /etc/gitlab-runner/config.toml
Modify the following keys:
concurrent = 4 [[runners]] environment = ["DOCKER_TLS_CERTDIR="] [runners.docker] privileged = true
concurrent key controls how many parallel jobs can be executed by this Runner, so the value depends on how many CPUs and RAM you have available. Setting the
environment key resolves the issue that pops up if dind image is used in the pipeline. The
privileged key set to
true also resolves the issue with
dind. Remember that enabling this flag can cause serious security risks which are well highlighted in this blog post – use it only if you fully control the pipelines.
After configuration was changed, restart the Runner so it can take effect:
$ sudo gitlab-runner restart
In order to speed up pipeline executions, configure the Runner to use S3 object store as a cache. Apply the following snippet to the
[[runners]] [runners.cache] Type = "s3" Path = "jobs-cache" Shared = true [runners.cache.s3] ServerAddress = "s3.amazonaws.com" AccessKey = "[ACCESS KEY]" SecretKey = "[SECRET KEY]" BucketName = "nubisoft-gitlab-ci-runners-cache" BucketLocation = "eu-central-1" Insecure = false
SECRET_KEY with values obtained in AWS console when created dedicated IAM user.
Private Docker Registry
To pull Docker images from private GitLab Registry in CI pipelines login to the registry with the following command:
$ sudo docker login registry.gitlab.com
As a credentials use Deploy token created for a group. Remember to assign
And that’s it! Your fresh Runner is up and running, ready to accept jobs.