前段时间搞了下Drone，今天刚好有空，记录下别忘记了。 这些基础设施都是用Docker来搭建的，不明白的估计要先补一下。
官网链接：
本次学习的目的
- 部署Spring Boot应用(单机 非k8s)
- 自动部署前端公共模块到Npm私库Verdaccio
Drone Gogs Verdaccio搭建
先看看目录结构：
image.png
step 1
version: "3"
services:
gogs:
image: gogs/gogs:latest
container_name: gogs
ports:
- "10022:22"
- "3000:3000"
environment:
TZ: Asia/Shanghai
volumes:
- ./data/gogs:/data
depends_on:
- mysql
links:
- mysql:mysql
mysql:
image: mysql:5.7
container_name: mysql
volumes:
- ./gogs/mysql:/var/lib/mysql
ports:
- 3308:3306
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
environment:
MYSQL_ROOT_PASSWORD: gogs123
MYSQL_DATABASE: gogs
MYSQL_USER: gogs
MYSQL_PASSWORD: gogs123456
TZ: Asia/Shanghai
verdaccio:
image: verdaccio/verdaccio:latest
container_name: verdaccio
volumes:
- ./verdaccio/conf:/verdaccio/conf
- ./verdaccio/storage:/verdaccio/storage
- ./verdaccio/plugins:/verdaccio/plugins
ports:
- 4873:4873
drone-server:
image: drone/drone:latest
container_name: drone_server
restart: always
ports:
- "9500:80"
- 8843:443
volumes:
- ./drone:/var/lib/drone/
environment:
- DRONE_OPEN=true
- DRONE_DEBUG=true
- DRONE_SERVER_PROTO=http
- DRONE_SERVER_HOST=192.168.0.225:9500
- DRONE_GIT_ALWAYS_AUTH=false
- DRONE_GOGS=true
- DRONE_GOGS_SKIP_VERIFY=false
- DRONE_GOGS_SERVER=http://192.168.0.225:3000
- DRONE_PROVIDER=gogs
- DRONE_RPC_SECRET=drone-sec-aaaaaaaaaa
- DRONE_SECRET=drone-sec-ffffffff
- DRONE_USER_CREATE=username:drone_admin,admin:true
- TZ=Asia/Shanghai
drone-runner-docker:
image: drone/drone-runner-docker:1.4.0
container_name: drone_runner_docker
restart: always
ports:
- "1230:3000"
depends_on:
- drone-server
- drone-registry-plugin
volumes:
- /var/run/docker.sock:/var/run/docker.sock
# - //./pipe/docker_engine://./pipe/docker_engine
environment:
- DRONE_RPC_HOST=192.168.0.225:9500
- DRONE_RPC_PROTO=http
- DRONE_RPC_SECRET=drone-sec-aaaaaaaaaa
- DRONE_RUNNER_CAPACITY=1
- DRONE_RUNNER_NAME=docker-runner
- DRONE_REGISTRY_PLUGIN_ENDPOINT=http://192.168.0.225:3030
- DRONE_REGISTRY_PLUGIN_TOKEN=drone_registry_xxxxx
- TZ=Asia/Shanghai
drone-registry-plugin:
image: drone/registry-plugin
container_name: drone-registry-plugin
ports:
- 3030:3000
environment:
- DRONE_DEBUG=true
- DRONE_SECRET=drone_registry_xxxxx
- DRONE_CONFIG_FILE=/etc/registry_config.yml
volumes:
- ./registry.yml:/etc/registry_config.yml
上为docker-compose.yml，说明：
首先我本地的的ip为192.168.0.225，环境为Linux，Docker容器一些重要的东西（数据&配置）能挂载出来的就挂载出来，不然容器销毁，东西就没得了。
- Gogs，一个git仓库，依赖于下面的mysql存储。WEB端口为3000
- Mysql，这个就不用多说了，端口为3308
- Verdaccio，Npm私库，端口为4873
- Drone Server，Drone服务端，端口为9500，注意一些环境变量的设置，因为需要一些服务的通信，需要一些secret或者token认证，仔细校对一下。这里有一个DRONE_USER_CREATE变量注意一下，这个东西就是标识谁是管理员的，不是管理员的用户有些东西是看不了的。
- Drone runner，Drone执行端，Drone可以有多个runner，runner还有不同类型的。这里使用一个Docker类型的runner。这里依赖了一个drone-registry-plugin，这个东西就是用来设置runner中docker的镜像地址，不然速度超慢。
step 2
#
# This is the config file used for the docker images.
# It allows all users to do anything, so don't use it on production systems.
#
# Do not configure host and port under `listen` in this file
# as it will be ignored when using docker.
# see https://verdaccio.org/docs/en/docker#docker-and-custom-port-configuration
#
# Look here for more config file examples:
# https://github.com/verdaccio/verdaccio/tree/master/conf
#
# path to a directory with all packages
storage: /verdaccio/storage/data
# path to a directory with plugins to include
plugins: /verdaccio/plugins
web:
# WebUI is enabled as default, if you want disable it, just uncomment this line
#enable: false
title: Verdaccio
# comment out to disable gravatar support
# gravatar: false
# by default packages are ordercer ascendant (asc|desc)
# sort_packages: asc
auth:
htpasswd:
file: /verdaccio/storage/htpasswd
# Maximum amount of users allowed to register, defaults to "+infinity".
# You can set this to -1 to disable registration.
# max_users: 1000
# a list of other known repositories we can talk to
uplinks:
npmjs:
url: https://registry.npm.taobao.org/
packages:
'@*/*':
# scoped packages
access: $all
publish: $authenticated
# proxy: npmjs
'**':
# allow all users (including non-authenticated users) to read and
# publish all packages
#
# you can specify usernames/groupnames (depending on your auth plugin)
# and three keywords: "$all", "$anonymous", "$authenticated"
access: $all
# allow all known users to publish/publish packages
# (anyone can register by default, remember?)
publish: $authenticated
# if package is not available locally, proxy requests to 'npmjs' registry
# proxy: npmjs
middlewares:
audit:
enabled: true
# log settings
logs:
- { type: stdout, format: pretty, level: http }
#- {type: file, path: verdaccio.log, level: info}
#experiments:
# # support for npm token command
# token: false
/verdaccio/conf/config.yaml 说明：
这个文件就是verdaccio的配置文件，详情可以去看看官网。
这个有一个注意的点，verdaccio目录下面的东西会挂载到镜像容器里面，里面会有一些文件操作，所以要给权限。
sudo chown -R 10001:65533 verdaccio
step 3
- address: https://jif70l5j.mirror.aliyuncs.com
username:
password:
- address: https://hub-mirror.c.163.com/
username:
password:
registry.yml,docker镜像配置
step 4 (settings.xml,可略过spring的私库，后面会用到)
<settings xmlns="http://maven.apache.org/SETTINGS/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">
<localRepository>/usr/share/maven/ref/repository</localRepository>
<mirrors>
<mirror>
<id>nexus-tout</id>
<!-- *号代表所有仓库，此处也可以单独设置，以逗号隔开 -->
<mirrorOf>*</mirrorOf>
<name>Nexus3 tout</name>
<url>http://maven.xxxxx.com/content/groups/public/</url>
</mirror>
</mirrors>
<pluginGroups>
<pluginGroup>org.apache.maven.plugins</pluginGroup>
<pluginGroup>org.codehaus.mojo</pluginGroup>
</pluginGroups>
</settings>
step 5
sudo chown -R 10001:65533 verdaccio
sudo cp settings.xml /data/maven/settings.xml
sudo mkdir -p /data/npm/cache/node_modules
sudo docker-compose up
执行命令，启动服务，推荐本地的docker镜像也设置一下。
Harbor
这个东西直接就安装官网安装就行，
https://github.com/goharbor/harbor/releases，我用的离线版的，配置好harbor.yml，就可以install.sh运行。实际这个东西也是通过docker-compose来运行的。可以整合到上面那个docker-compose，不过有些麻烦。
# Configuration file of Harbor
# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname: 192.168.0.225
# http related config
http:
# port for http, default is 80. If https enabled, this port will redirect to https port
port: 8050
# https related config
# https:
# https port for harbor, default is 443
# port: 443
# The path of cert and key files for nginx
# certificate: /your/certificate/path
# private_key: /your/private/key/path
# # Uncomment following will enable tls communication between all harbor components
# internal_tls:
# # set enabled to true means internal tls is enabled
# enabled: true
# # put your cert and key files on dir
# dir: /etc/harbor/tls/internal
# Uncomment external_url if you want to enable external proxy
# And when it enabled the hostname will no longer used
# external_url: https://reg.mydomain.com:8433
# The initial password of Harbor admin
# It only works in first time to install harbor
# Remember Change the admin password from UI after launching Harbor.
harbor_admin_password: Harbor12345
# Harbor DB configuration
database:
# The password for the root user of Harbor DB. Change this before any production use.
password: root123
# The maximum number of connections in the idle connection pool. If it <=0, no idle connections are retained.
max_idle_conns: 50
# The maximum number of open connections to the database. If it <= 0, then there is no limit on the number of open connections.
# Note: the default number of connections is 1024 for postgres of harbor.
max_open_conns: 1000
# The default data volume
data_volume: /data/harbor
# Harbor Storage settings by default is using /data dir on local filesystem
# Uncomment storage_service setting If you want to using external storage
# storage_service:
# # ca_bundle is the path to the custom root ca certificate, which will be injected into the truststore
# # of registry's and chart repository's containers. This is usually needed when the user hosts a internal storage with self signed certificate.
# ca_bundle:
# # storage backend, default is filesystem, options include filesystem, azure, gcs, s3, swift and oss
# # for more info about this configuration please refer https://docs.docker.com/registry/configuration/
# filesystem:
# maxthreads: 100
# # set disable to true when you want to disable registry redirect
# redirect:
# disabled: false
# Clair configuration
clair:
# The interval of clair updaters, the unit is hour, set to 0 to disable the updaters.
updaters_interval: 12
# Trivy configuration
trivy:
# ignoreUnfixed The flag to display only fixed vulnerabilities
ignore_unfixed: false
# skipUpdate The flag to enable or disable Trivy DB downloads from GitHub
#
# You might want to enable this flag in test or CI/CD environments to avoid GitHub rate limiting issues.
# If the flag is enabled you have to manually download the `trivy.db` file and mount it in the
# /home/scanner/.cache/trivy/db/trivy.db path.
skip_update: false
#
# insecure The flag to skip verifying registry certificate
insecure: false
# github_token The GitHub access token to download Trivy DB
#
# Trivy DB contains vulnerability information from NVD, Red Hat, and many other upstream vulnerability databases.
# It is downloaded by Trivy from the GitHub release page https://github.com/aquasecurity/trivy-db/releases and cached
# in the local file system (/home/scanner/.cache/trivy/db/trivy.db). In addition, the database contains the update
# timestamp so Trivy can detect whether it should download a newer version from the Internet or use the cached one.
# Currently, the database is updated every 12 hours and published as a new release to GitHub.
#
# Anonymous downloads from GitHub are subject to the limit of 60 requests per hour. Normally such rate limit is enough
# for production operations. If, for any reason, it's not enough, you could increase the rate limit to 5000
# requests per hour by specifying the GitHub access token. For more details on GitHub rate limiting please consult
# https://developer.github.com/v3/#rate-limiting
#
# You can create a GitHub token by following the instuctions in
# https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line
#
# github_token: xxx
jobservice:
# Maximum number of job workers in job service
max_job_workers: 10
notification:
# Maximum retry count for webhook job
webhook_job_max_retry: 10
chart:
# Change the value of absolute_url to enabled can enable absolute url in chart
absolute_url: disabled
# Log configurations
log:
# options are debug, info, warning, error, fatal
level: info
# configs for logs in local storage
local:
# Log files are rotated log_rotate_count times before being removed. If count is 0, old versions are removed rather than rotated.
rotate_count: 50
# Log files are rotated only if they grow bigger than log_rotate_size bytes. If size is followed by k, the size is assumed to be in kilobytes.
# If the M is used, the size is in megabytes, and if G is used, the size is in gigabytes. So size 100, size 100k, size 100M and size 100G
# are all valid.
rotate_size: 200M
# The directory on your host that store log
location: /var/log/harbor
# Uncomment following lines to enable external syslog endpoint.
# external_endpoint:
# # protocol used to transmit log to external endpoint, options is tcp or udp
# protocol: tcp
# # The host of external endpoint
# host: localhost
# # Port of external endpoint
# port: 5140
#This attribute is for migrator to detect the version of the .cfg file, DO NOT MODIFY!
_version: 2.0.0
# Uncomment external_database if using external database.
# external_database:
# harbor:
# host: harbor_db_host
# port: harbor_db_port
# db_name: harbor_db_name
# username: harbor_db_username
# password: harbor_db_password
# ssl_mode: disable
# max_idle_conns: 2
# max_open_conns: 0
# clair:
# host: clair_db_host
# port: clair_db_port
# db_name: clair_db_name
# username: clair_db_username
# password: clair_db_password
# ssl_mode: disable
# notary_signer:
# host: notary_signer_db_host
# port: notary_signer_db_port
# db_name: notary_signer_db_name
# username: notary_signer_db_username
# password: notary_signer_db_password
# ssl_mode: disable
# notary_server:
# host: notary_server_db_host
# port: notary_server_db_port
# db_name: notary_server_db_name
# username: notary_server_db_username
# password: notary_server_db_password
# ssl_mode: disable
# Uncomment external_redis if using external Redis server
# external_redis:
# host: redis
# port: 6379
# password:
# # db_index 0 is for core, it's unchangeable
# registry_db_index: 1
# jobservice_db_index: 2
# chartmuseum_db_index: 3
# clair_db_index: 4
# trivy_db_index: 5
# idle_timeout_seconds: 30
# Uncomment uaa for trusting the certificate of uaa instance that is hosted via self-signed cert.
# uaa:
# ca_file: /path/to/ca
# Global proxy
# Config http proxy for components, e.g. http://my.proxy.com:3128
# Components doesn't need to connect to each others via http proxy.
# Remove component from `components` array if want disable proxy
# for it. If you want use proxy for replication, MUST enable proxy
# for core and jobservice, and set `http_proxy` and `https_proxy`.
# Add domain to the `no_proxy` field, when you want disable proxy
# for some special registry.
proxy:
http_proxy:
https_proxy:
no_proxy:
components:
- core
- jobservice
- clair
- trivy
这个是我的配置。端口为8050
账号准备
verdaccio账号：
npm adduser --registry http://192.168.0.225:4873
harbor账号：
我们还是假装遵循权限最小原则，利用admin账号创建一个子账号。
image.png
然后利用子账号创建一个项目，这个项目不是一个镜像，项目下面是可以放置很多镜像的。
image.png
这个里面就是放置镜像的。
开始部署
访问gogs(192.168.0.225:3000)，如果第一次要让你配置，填一下就行。部署肯定需要项目，把要部署的项目先提交上去。然后去仓库设置里面：
image.png
没有这个就手动填一下，我这个不是手填的，我这个好像是在drone那边点了一个自动同步，没得截图了，emmmm...
然后去看drone那边（192.168.0.225:9500），有没有对应的仓库，大概是这个样子
image.png
部署前端公共库
image.png
image.png
上面有两个配置，一个是部署的配置，一个是部署变量（一些敏感信息）的配置。
最后一步编写 .drone.yml，在项目的根目录下创建此文件。
kind: pipeline
name: 前端公共库
type: docker
steps:
- name: npm-auth
image: robertstettner/drone-npm-auth
settings:
username:
from_secret: npm_username
password:
from_secret: npm_password
email: 121116111@qq.com
registry: "http://192.168.0.225:4873"
- name: build
image: node:12.16.1
commands:
- node -v
- npm -v
- yarn -v
- yarn install
- yarn lib
- npm config set registry http://192.168.0.225:4873
- npm publish
- name: dingtalk
image: lddsb/drone-dingtalk-message
settings:
token:
from_secret: dingtalk_token
secret:
from_secret: dingtalk_secret
type: markdown
when:
status:
- failure
- success
trigger:
branch:
- master
event:
- push
大概说明一下
这个部署的触发条件是master发生push，然后按照步骤执行。详细的参数设置可以看看文档。
1.认证私库
2.打包并上传到私库
3.钉钉通知（可以弄成其他的）
image.png
spring boot
kind: pipeline
type: docker
name: Credit管理-测试
steps:
- name: restore-cache
image: drillster/drone-volume-cache
volumes:
- name: npm-cache
path: /cache
settings:
restore: true
mount:
- ./credit-manager-ui/node_modules
- name: front-build
image: node:12.16.1
commands:
- yarn config set registry http://192.168.0.225:4873
- cd ./credit-manager-ui
- yarn install
- yarn build:test
- name: rebuild-cache
image: drillster/drone-volume-cache
volumes:
- name: npm-cache
path: /cache
settings:
rebuild: true
mount:
- ./credit-manager-ui/node_modules
- name: web-build
image: maven:3.6.3-jdk-8
volumes:
- name: maven-cache # The Volume's name
path: /root/.m2
commands:
- mvn clean install -e -U -pl credit-manager-web -am
- cd ./credit-manager-web/target
- ls
- name: build-image-and-push
image: plugins/docker
settings:
repo: 192.168.0.225:8050/xxxx-dev/credit-manager
tags:
- ${DRONE_BUILD_NUMBER}
insecure: true
use_cache: true
mirror: https://jif70l5j.mirror.aliyuncs.com
registry: 192.168.0.225:8050
username:
from_secret: harbor_name
password:
from_secret: harbor_password
dockerfile: ./Dockerfile_credit
- name: ssh-deploy
image: appleboy/drone-ssh
settings:
host:
from_secret: dev_host
username:
from_secret: dev_username
password:
from_secret: dev_password
command_timeout: 3m
port: 22
script:
- sudo docker stop credit-manager
- sudo docker rm credit-manager
- sudo docker run -itd --name credit-manager -e HOST='192.168.0.225' -p 9001:9001 192.168.0.225:8050/xxxx-dev/credit-manager:${DRONE_BUILD_NUMBER}
- bash <(curl -s -S -L https://images.xxxx.com/sh/check302.sh) 'localhost:9001/credit-manager'
- ls
trigger:
branch:
- develop
volumes:
- name: npm-cache
host:
path: /data/npm
- name: maven-cache # The name use in this pipeline,
host:
path: /data/maven # The path be used in the host.
简单说一些这个项目，这个项目是前后端一起打包的，所以能看到前端的打包。
1.恢复npm的缓存
2.前端打包
3.更新缓存
4.后端打包（jar，里面也有前端）
5.通过Dockerfile构建镜像并上传到harbor
6.通过ssh链接服务器，拉取镜像，执行镜像，然后检测服务的状态。
第6步，如果有集群可以用集群部署。
Dockerfile长这个样子
FROM openjdk:8-jdk-alpine
ENV LANG C.UTF-8
COPY ./credit-manager-web/target/credit-manager-web.jar app.jar
EXPOSE 9001
ENTRYPOINT java -Xms512m -Xmx512m -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m -jar /app.jar --spring.profiles.active=default --server.port=9001
image.png
总结
这个东西断断续续搞了一周，虽然现在没用上，但是值得了解一下。其实搭建基础设置这一块还挺简单的，网上资料也多，有些滞后的就去看官网。主要还是写部署那一块比较费时间😂。
