Setup a Gitlab Runner on your own Server to run your jobs that gets triggered from Gitlab CI

From our previous post, we went through the setup on setting up a Basic CI Pipeline on Gitlab, in conjunction with Gitlab CI which coordinates your jobs, where we used the Shared Runners, which runs your jobs on Gitlab's Infrastructure.

In Gitlab, you have Shared Runners and your Own Runners, which is used to run your jobs and send the results back to GitLab.

In this tutorial we will Setup a Server with gitlab-runner and Docker on Ubuntu and then Setup a Basic Pipeline to Utilize your Gitlab Runner.

Setup Docker

Install Docker:

$ sudo apt update && sudo apt upgrade -y
$ sudo apt-get install apt-transport-https ca-certificates curl software-properties-common -y
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

$ sudo apt update
$ sudo apt install docker-ce -y
$ docker run hello-world

Install and Setup Gitlab Runner

This setup is intended for Linux 64bit, for other distributions, have a look at their docs

Install the Runner:

$ wget -O /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64
$ chmod +x /usr/local/bin/gitlab-runner
$ useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
$ gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
$ gitlab-runner start

Register the Runner. The Gitlab-CI Token is available in your CI/CD Settings panel from the UI: https://gitlab.com/<account>/<repo>/settings/ci_cd

$ gitlab-runner register
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
https://gitlab.com/

Please enter the gitlab-ci token for this runner:
__masked__

Please enter the gitlab-ci description for this runner:
[my-runner]: my-runner

Please enter the gitlab-ci tags for this runner (comma separated):
my-runner,foobar
Registering runner... succeeded                     runner=66m_339h

Please enter the executor: docker-ssh+machine, docker, docker-ssh, parallels, shell, ssh, virtualbox, docker+machine, kubernetes:
docker

Please enter the default Docker image (e.g. ruby:2.1):
alpine:latest

Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

Verify the Status and check if Docker and Gitlab Runner is enabled on startup:

$ gitlab-runner status
Runtime platform                                    arch=amd64 os=linux pid=30363 revision=7f00c780 version=11.5.1
gitlab-runner: Service is running!

$ systemctl is-enabled gitlab-runner
enabled

$ systemctl is-enabled docker
enabled

Gitlab-CI Config for Shared Runners

If you would like to use the shared runners that Gitlab Offers, the .gitlab-ci.yml config will look like this:

stages:
  - build
  - test

build:
  stage: build
  script:
    - echo "this is building"
    - hostname
    - mkdir builds
    - touch builds/data.txt
    - echo "true" > builds/data.txt
  artifacts:
    paths:
      - builds/

test:
  stage: test
  script:
    - echo "this is testing"
    - hostname
    - test -f builds/data.txt
    - grep "true" builds/data.txt

Gitlab-CI Config for your own Gitlab Runner

Gitlab utilizes the tags that was specified on registration to determine where the jobs gets executed on, for more information on this, have a look at their docs

The .gitlab-ci.yml config for using your gitlab runner:

stages:
  - build
  - test

build:
  stage: build
  tags:
    - my-runner
  script:
    - echo "this is building"
    - hostname
    - mkdir builds
    - touch builds/data.txt
    - echo "true" > builds/data.txt
  artifacts:
    paths:
      - builds/

test:
  stage: test
  tags:
    - my-runner
  script:
    - echo "this is testing"
    - hostname
    - test -f builds/data.txt
    - grep "true" builds/data.txt

Trigger and Check Docker

Commit the config to master, let your pipeline run their jobs upon completion have a look at docker on your server for the containers that the jobs ran on:

$ docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS                          PORTS               NAMES
04292a78de0b        c04b8be95e1e        "gitlab-runner-cache.."  About a minute ago   Exited (0) About a minute ago                       runner-xx-project-xx-concurrent-0-cache-3cxx0
49b1b3c4adf9        c04b8be95e1e        "gitlab-runner-cache.."  About a minute ago   Exited (0) About a minute ago                       runner-xx-project-xx-concurrent-0-cache-6cxxa
422b23191e8c        hello-world         "/hello"                 24 minutes ago       Exited (0) 24 minutes ago                           wizardly_meninsky

As we know each job gets executed in different containers, you can see from the output above that there was 2 different containers for the 2 jobs that was specified in our pipeline.

Resources:

Thank You

Thanks for reading, if you like my content, feel free to check out my website, and subscribe to my newsletter or follow me at @ruanbekker on Twitter.

Linktree: https://go.ruan.dev/links
Patreon: https://go.ruan.dev/patreon