Gitlab CI/CD with gitlab-runner in docker mode

1. Install and start the gitlab-runner container

通过docker拉取gitlab-runner镜像，启动了一个gitlab-runner容器。假设你的私有化部署的gitlab网站做了一个host映射，例如例子中的gitlab.oa.com，那么需要通过--add-host方式将该host信息加入到gitlab-runner这个容器的/etc/hosts中。只有这样设置后，才能够完成gitlab-runner的注册。

docker run -d \
  --name gitlab-runner \
  --add-host gitlab.oa.com:192.168.146.60 \
  --restart always \
  --volume /data/gitlab-runner/config:/etc/gitlab-runner \
  --volume /data/gitlab-runner/build:/build \
  --volume /var/run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:latest

2. Register a runner for the gitlab site

注册时可以通过这样的方式创建一个docker executor，这里用的是Java的例子，使用了一个maven3的image，这个image中带了jdk-8. 通过设置--tag-list "java,spring"的方式，只要我们的Java项目的.gitlab-ci.yml中tags包含java或者spring，就可以分配到这个gitlab-runner上执行。注意要根据你从gitlab管理页面创建的token内容替换--registration-token的内容。

docker exec -it gitlab-runner gitlab-runner register --non-interactive --url "http://gitlab.oa.com/" --registration-token "token from your gitlab admin page" --description "java runner"  --tag-list "java,spring" --run-untagged --locked="false" --executor "docker" --docker-volumes /data/gitlab-runner/ws:/share:rw --docker-image maven:3-jdk-8

执行完成这样的命令后，将在/data/gitlab-runner/config/中创建一个config.toml文件，内容如下

➜  ~ cat /data/gitlab-runner/config/config.toml
concurrent = 1
check_interval = 0


[session_server]
  session_timeout = 1800


[[runners]]
  name = "java runner"
  url = "http://gitlab.oa.com/"
  token = "your token"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.docker]
    tls_verify = false
    image = "maven:3-jdk-8"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/data/gitlab-runner/ws:/share:rw", "/cache"]
    shm_size = 0

因为我们使用了自定义的hosts，还需要为docker in docker中的docker增加hosts信息，否则会导致无法解析gitlab.oa.com域名的问题。

2.1 extra_hosts and volumes setting

从官方文档 runner.docker settings中，我们了解到可以增加extra_hosts等参数来影响docker中的docker的执行参数。

➜  ~ cat /data/gitlab-runner/config/config.toml
concurrent = 1
check_interval = 0


[session_server]
  session_timeout = 1800


[[runners]]
  name = "java runner"
  url = "http://gitlab.oa.com/"
  token = "your token"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.docker]
    tls_verify = false
    image = "maven:3-jdk-8"
    extra_hosts = ["gitlab.oa.com:192.168.146.60"] # add this extra_hosts for docker in docker
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/data/gitlab-runner/ws:/share:rw", "/data/share/maven:/root/.m2:rw", "/cache"]  # share the maven repository and setting
    shm_size = 0

2.2 Example project with .gitlab-ci.yml

我们创建一个maven项目，目录结构如下，包括了两个lib，三个app，共5个module。

├─app                                                      
│  ├─app-signal-processor                                                                  
│  ├─watchlist-event-processor                                                         
│  └─watchlist-loader                                        
└─lib                                                      
    ├─common                                                                          
    ├─parent                                               
    └─publisher
pom.xml
.gitlab-ci.yml
.gitignore

其中.gitlab-ci.yml文件就是gitlab ci/cd的配置文件，有这个文件就能够定义你的DevOps流程。这里仅仅简单定义一个commit就触发如下流程

image-20200819125933954

配置文件内容是：

# This file is a template, and might need editing before it works on your project.
# Build JAVA applications using Apache Maven (http://maven.apache.org)
# For docker image tags see https://hub.docker.com/_/maven/
#
# For general lifecycle information see https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html

# This template will build and test your projects
# * Caches downloaded dependencies and plugins between invocation.
# * Verify but don't deploy merge requests.
# * Deploy built artifacts from master branch only.

variables:
  # This will suppress any download for dependencies and plugins or upload messages which would clutter the console log.
  # `showDateTime` will show the passed time in milliseconds. You need to specify `--batch-mode` to make this work.
  MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
  # As of Maven 3.3.0 instead of this you may define these options in `.mvn/maven.config` so the same config is used
  # when running from the command line.
  # `installAtEnd` and `deployAtEnd` are only effective with recent version of the corresponding plugins.
  MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"


# Cache downloaded dependencies and plugins between builds.
# To keep cache across branches add 'key: "$CI_JOB_NAME"'
cache:
  paths:
    - .m2/repository

# This will only validate and compile stuff and run e.g. maven-enforcer-plugin.
# Because some enforcer rules might check dependency convergence and class duplications
# we use `test-compile` here instead of `validate`, so the correct classpath is picked up.
validate: 
  stage: build
  tags:
    - java
  script:
    - 'mvn $MAVEN_CLI_OPTS test-compile'

# For merge requests do not `deploy` but only run `verify`.
# See https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
testing:
  stage: test
  tags:
    - java
  script:
    - 'mvn $MAVEN_CLI_OPTS verify test'
  artifacts:
    expire_in: 2 weeks
    reports:
      junit: ./**/target/surefire-reports/TEST-*.xml



# For `master` branch run `mvn deploy` automatically.
# Here you need to decide whether you want to use JDK7 or 8.
# To get this working you need to define a volume while configuring your gitlab-ci-multi-runner.
# Mount your `settings.xml` as `/root/.m2/settings.xml` which holds your secrets.
# See https://maven.apache.org/settings.html
deploy:jdk8:
  # Use stage test here, so the pages job may later pickup the created site.
  stage: test
  tags:
    - java
  script:
    - 'mvn $MAVEN_CLI_OPTS package -DskipTests'
    #- 'mvn $MAVEN_CLI_OPTS deploy site site:stage'
  only:
    - master
  # Archive up the built documentation site.
  artifacts:
    expire_in : 2 hrs
    paths:
      - ./**/target/*.jar

我们会在每次deploy阶段将成功打包的内容暂时存起来，供后续的自动化集成测试使用。

image-20200819130047137

image-20200819130119054