This is the first post from our series<\/a> about building robust CI\/CD pipelines for Node-based projects, wherein we will focus on how to prepare the infrastructure for pipeline execution.<\/p>\n\n\n\n In GitLab<\/a> CI a Runner is a dedicated component<\/a> that is responsible for executing jobs that are part of a given pipeline. So essentially, it’s a piece of code which is running on some VPS or VM and can build, test and deploy your code. As a part of a free plan, GitLab gives you Shared Runners for free – at least for first 2000 minutes per month, but as you may suspect they are not very powerful in terms of CPU and RAM (see here for detailed specification). If you want to reduce the time needed to execute CI\/CD pipeline, wise option would be to spin up a few VPS in cloud providers of your choice (we use Digital Ocean<\/a> and VPSDime<\/a>) and use them instead of shared machines on GitLab.<\/p>\n\n\n\n Let’s see how to do that!<\/p>\n\n\n\n In order to install Runner on Linux machine, first add the GitLab package repository and then install it:<\/p>\n\n\n\n Because we will be using Docker please ensure it is also installed<\/a>.<\/p>\n\n\n\n When Runner is installed on a target machine it can be now registered. Preferably, register it as a group Runner<\/a> so that it can pickup jobs from any project under that group (more about project groups can be found in the documentation<\/a>).<\/p>\n\n\n\n Before starting the registration process, go to GitLab and open your group CI\/CD settings page<\/a>, expand Runners<\/em> and go to Set up a group Runner manually<\/em> section. Copy URL and token.<\/p>\n\n\n\n Now, on a target machine register the Runner<\/a> by issuing the following command in terminal:<\/p>\n\n\n\n Enter the copied URL and token, optionally specify tags<\/a> and use docker<\/a> as an Executor<\/em>.<\/p>\n\n\n\n Before Runner can execute any job it must be first configured. To set up it’s parameters issue the following command in the terminal:<\/p>\n\n\n\n Modify the following keys:<\/p>\n\n\n\n The After configuration was changed, restart the Runner so it can take effect:<\/p>\n\n\n\n In order to speed up pipeline executions, configure the Runner to use S3<\/a> object store as a cache. Apply the following snippet to the Replace To pull Docker images from private GitLab Registry<\/a> in CI pipelines login to the registry with the following command:<\/p>\n\n\n\nIntroduction<\/h3>\n\n\n\n
Installation<\/h3>\n\n\n\n
$ curl -L https:\/\/packages.gitlab.com\/install\/repositories\/runner\/gitlab-runner\/script.deb.sh | sudo bash\n$ sudo apt-get install gitlab-runner<\/pre>\n\n\n\n
Registration<\/h3>\n\n\n\n
$ sudo gitlab-runner register<\/pre>\n\n\n\n
Configuration<\/h3>\n\n\n\n
$ sudo nano \/etc\/gitlab-runner\/config.toml<\/pre>\n\n\n\n
concurrent = 4\n[[runners]]\n environment = [\"DOCKER_TLS_CERTDIR=\"]\n [runners.docker]\n privileged = true<\/pre>\n\n\n\n
concurrent<\/code> key controls how many parallel jobs can be executed by this Runner, so the value depends on how many CPUs and RAM you have available. Setting the environment<\/code> key resolves the issue<\/a> that pops up if dind<\/a> image is used in the pipeline. The privileged<\/code> key set to true<\/code> also resolves the issue<\/a> with dind<\/code>. Remember that enabling this flag can cause serious security risks which are well highlighted in this<\/a> blog post – use it only if you fully control the pipelines.<\/p>\n\n\n\n$ sudo gitlab-runner restart<\/pre>\n\n\n\n
Cache<\/h4>\n\n\n\n
\/etc\/gitlab-runner\/config.toml<\/code> file:<\/p>\n\n\n\n[[runners]]\n [runners.cache]\n Type = \"s3\"\n Path = \"jobs-cache\"\n Shared = true\n [runners.cache.s3]\n ServerAddress = \"s3.amazonaws.com\"\n AccessKey = \"[ACCESS KEY]\"\n SecretKey = \"[SECRET KEY]\"\n BucketName = \"nubisoft-gitlab-ci-runners-cache\"\n BucketLocation = \"eu-central-1\"\n Insecure = false<\/pre>\n\n\n\n
ACCESS_KEY<\/code> and SECRET_KEY<\/code> with values obtained in AWS console when created dedicated IAM user.<\/p>\n\n\n\nPrivate Docker Registry<\/h4>\n\n\n\n
$ sudo docker login registry.gitlab.com<\/pre>\n\n\n\n