前面我们手工部署了redis-cluster集群，可以看到步骤烦多，现在使用docker的方式部署，非常简单，共分为二步。
参考镜像https://github.com/publicisworldwide/docker-stacks/tree/master/oracle-linux/environments/storage/redis-cluster
Docker方式部署redis-cluster步骤
1、redis容器初始化
2、redis容器集群配置
一、redis容器初始化
容器初始化，使用docker-compose方式，先创建一个docker-compose.yml文件，内容如下：
version: '3'
services:
redis1:
image: publicisworldwide/redis-cluster
network_mode: host
restart: always
volumes:
- /data/redis/8001/data:/data
environment:
- REDIS_PORT=8001
redis2:
image: publicisworldwide/redis-cluster
network_mode: host
restart: always
volumes:
- /data/redis/8002/data:/data
environment:
- REDIS_PORT=8002
redis3:
image: publicisworldwide/redis-cluster
network_mode: host
restart: always
volumes:
- /data/redis/8003/data:/data
environment:
- REDIS_PORT=8003
redis4:
image: publicisworldwide/redis-cluster
network_mode: host
restart: always
volumes:
- /data/redis/8004/data:/data
environment:
- REDIS_PORT=8004
redis5:
image: publicisworldwide/redis-cluster
network_mode: host
restart: always
volumes:
- /data/redis/8005/data:/data
environment:
- REDIS_PORT=8005
redis6:
image: publicisworldwide/redis-cluster
network_mode: host
restart: always
volumes:
- /data/redis/8006/data:/data
environment:
- REDIS_PORT=8006
这里引用了别人的一个镜像publicisworldwide/redis-cluster，方便快捷。
这里使用host(主机)网络模式，把redis数据挂载到本机目录/data/redis/800*下。
若不想使用host模式，也可以把network_mode去掉，但就要加ports映射。
redis-cluster的节点端口共分为2种，一种是节点提供服务的端口，如6379；一种是节点间通信的端口，固定格式为：10000+6379。
version: '3'
services:
redis1:
image: publicisworldwide/redis-cluster
restart: always
volumes:
- /data/redis/8001/data:/data
environment:
- REDIS_PORT=8001
ports:
- '8001:8001' #服务端口
- '18001:18001' #集群端口
redis2:
image: publicisworldwide/redis-cluster
restart: always
volumes:
- /data/redis/8002/data:/data
environment:
- REDIS_PORT=8002
ports:
- '8002:8002'
- '18002:18002'
redis3:
image: publicisworldwide/redis-cluster
restart: always
volumes:
- /data/redis/8003/data:/data
environment:
- REDIS_PORT=8003
ports:
- '8003:8003'
- '18003:18003'
redis4:
image: publicisworldwide/redis-cluster
restart: always
volumes:
- /data/redis/8004/data:/data
environment:
- REDIS_PORT=8004
ports:
- '8004:8004'
- '18004:18004'
redis5:
image: publicisworldwide/redis-cluster
restart: always
volumes:
- /data/redis/8005/data:/data
environment:
- REDIS_PORT=8005
ports:
- '8005:8005'
- '18005:18005'
redis6:
image: publicisworldwide/redis-cluster
restart: always
volumes:
- /data/redis/8006/data:/data
environment:
- REDIS_PORT=8006
ports:
- '8006:8006'
- '18006:18006'
创建文件后，直接启动服务
窗口模式
docker-compose up
后台进程
docker-compose up -d
查看启动的进程
[root@localhost redis-cluster]# docker-compose ps
Name Command State Ports
----------------------------------------------------------------------
rediscluster_redis1_1 /usr/local/bin/entrypoint. ... Up
rediscluster_redis2_1 /usr/local/bin/entrypoint. ... Up
rediscluster_redis3_1 /usr/local/bin/entrypoint. ... Up
rediscluster_redis4_1 /usr/local/bin/entrypoint. ... Up
rediscluster_redis5_1 /usr/local/bin/entrypoint. ... Up
rediscluster_redis6_1 /usr/local/bin/entrypoint. ... Up
状态为Up，说明服务均已启动，镜像无问题。
注意：以上镜像不能设置永久密码，其实redis一般是内网访问，可以不需密码。
二、redis容器集群配置
上面只是启动了6个redis容器，并没有设置集群，通过下面的命令可以设置集群。
docker run --rm -it inem0o/redis-trib create --replicas 1 172.19.165.222:8001 172.19.165.222:8002 172.19.165.222:8003 172.19.165.222:8004 172.19.165.222:8005 172.19.165.222:8006
这里同样使用了另一个镜像inem0o/redis-trib，执行时会自动下载。
日志如下：
[root@localhost disconf]# docker run --rm -it inem0o/redis-trib create --replicas 1 172.19.165.222:8001 172.19.165.222:8002 172.19.165.222:8003 172.19.165.222:8004 172.19.165.222:8005 172.19.165.222:8006
Unable to find image 'inem0o/redis-trib:latest' locally
latest: Pulling from inem0o/redis-trib
a2b2998a36ab: Pull complete
a3ed95caeb02: Pull complete
46ab6b64c08e: Pull complete
3d82c3ac2025: Pull complete
Digest: sha256:0b89d25b387f70ef1c54605bdf061dd86e0833dbc0e2149390570b8b372278f8
Status: Downloaded newer image for inem0o/redis-trib:latest
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
172.19.165.222:8001
172.19.165.222:8002
172.19.165.222:8003
Adding replica 172.19.165.222:8004 to 172.19.165.222:8001
Adding replica 172.19.165.222:8005 to 172.19.165.222:8002
Adding replica 172.19.165.222:8006 to 172.19.165.222:8003
M: 67d9a6bb6875f3a0f9a53e5bb05ddeca8e656950 172.19.165.222:8001
slots:0-5460 (5461 slots) master
M: 206626063f31dcd7e69010ce13c258e786197f1e 172.19.165.222:8002
slots:5461-10922 (5462 slots) master
M: e9924018d95772b8ff535f6bc0605a6630837069 172.19.165.222:8003
slots:10923-16383 (5461 slots) master
S: 548f4e65fbab8dcde8aac187849d50983d68599d 172.19.165.222:8004
replicates 67d9a6bb6875f3a0f9a53e5bb05ddeca8e656950
S: 0a5c799c1f8fed083c50902639fc354e4c25aa8c 172.19.165.222:8005
replicates 206626063f31dcd7e69010ce13c258e786197f1e
S: 94e2530ddd05b0e9eb3e71a9616342bd6647a5e6 172.19.165.222:8006
replicates e9924018d95772b8ff535f6bc0605a6630837069
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join....
>>> Performing Cluster Check (using node 172.19.165.222:8001)
M: 67d9a6bb6875f3a0f9a53e5bb05ddeca8e656950 172.19.165.222:8001
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 94e2530ddd05b0e9eb3e71a9616342bd6647a5e6 172.19.165.222:8006@18006
slots: (0 slots) slave
replicates e9924018d95772b8ff535f6bc0605a6630837069
S: 0a5c799c1f8fed083c50902639fc354e4c25aa8c 172.19.165.222:8005@18005
slots: (0 slots) slave
replicates 206626063f31dcd7e69010ce13c258e786197f1e
M: e9924018d95772b8ff535f6bc0605a6630837069 172.19.165.222:8003@18003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
M: 206626063f31dcd7e69010ce13c258e786197f1e 172.19.165.222:8002@18002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 548f4e65fbab8dcde8aac187849d50983d68599d 172.19.165.222:8004@18004
slots: (0 slots) slave
replicates 67d9a6bb6875f3a0f9a53e5bb05ddeca8e656950
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
可以看到设置了3个Master，3个Slave，若想测试下集群，可以参考【Redis-Cluster集群】
只需二步就完成了集群的部署，但上面是使用了别人的镜像，若是要自己创建镜像该如何呢？
手动创建镜像
编写Dockerfile
#基础镜像
FROM redis
#修复时区
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN echo 'Asia/Shanghai' >/etc/timezone
#环境变量
ENV REDIS_PORT 8000
#ENV REDIS_PORT_NODE 18000
#暴露变量
EXPOSE $REDIS_PORT
#EXPOSE $REDIS_PORT_NODE
#复制
COPY entrypoint.sh /usr/local/bin/
COPY redis.conf /usr/local/etc/
#for config rewrite
RUN chmod 777 /usr/local/etc/redis.conf
RUN chmod +x /usr/local/bin/entrypoint.sh
#入口
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
#命令
CMD ["redis-server", "/usr/local/etc/redis.conf"]
编写shell文件entrypoint.sh
#!/bin/sh
#只作用于当前进程,不作用于其创建的子进程
set -e
#$0--Shell本身的文件名 $1--第一个参数 $@--所有参数列表
# allow the container to be started with `--user`
if [ "$1" = 'redis-server' -a "$(id -u)" = '0' ]; then
sed -i 's/REDIS_PORT/'$REDIS_PORT'/g' /usr/local/etc/redis.conf
chown -R redis . #改变当前文件所有者
exec gosu redis "$0" "$@" #gosu是sudo轻量级”替代品”
fi
exec "$@"
编写redis.conf
#端口
port REDIS_PORT
#开启集群
cluster-enabled yes
#配置文件
cluster-config-file nodes.conf
cluster-node-timeout 5000
#更新操作后进行日志记录
appendonly yes
#设置主服务的连接密码
# masterauth
#设置从服务的连接密码
# requirepass
注意：
- requirepass和masterauth不能启用，否则redis-trib创建集群失败。
- protected-mode 保护模式是禁止公网访问，但是不能设置密码和bind ip。
以上编写了3个文件，其实可以创建镜像了，不过也可以在docker-compose中自动创建。
编写docker-compose.yml文件
version: '3'
services:
redis1:
build: ./
restart: always
volumes:
- /data/redis/8001/data:/data
environment:
- REDIS_PORT=8001
ports:
- '8001:8001' #服务端口
- '18001:18001' #集群端口
redis2:
build: ./
restart: always
volumes:
- /data/redis/8002/data:/data
environment:
- REDIS_PORT=8002
ports:
- '8002:8002'
- '18002:18002'
redis3:
build: ./
restart: always
volumes:
- /data/redis/8003/data:/data
environment:
- REDIS_PORT=8003
ports:
- '8003:8003'
- '18003:18003'
redis4:
build: ./
restart: always
volumes:
- /data/redis/8004/data:/data
environment:
- REDIS_PORT=8004
ports:
- '8004:8004'
- '18004:18004'
redis5:
build: ./
restart: always
volumes:
- /data/redis/8005/data:/data
environment:
- REDIS_PORT=8005
ports:
- '8005:8005'
- '18005:18005'
redis6:
build: ./
restart: always
volumes:
- /data/redis/8006/data:/data
environment:
- REDIS_PORT=8006
ports:
- '8006:8006'
- '18006:18006'
创建镜像
窗口模式
docker-compose up
后台进程
docker-compose up -d
重新创建
docker-compose up --build
若创建成功，可以使用命令登录并查看集群信息。
[root@localhost src]# ./redis-cli -c -p 8003
127.0.0.1:8003> cluster info
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:1
cluster_size:0
cluster_current_epoch:0
cluster_my_epoch:0
cluster_stats_messages_sent:0
cluster_stats_messages_received:0
从上面的日志里看到，当前集群状态失败。
与上面相同，重新执行上面的第二步，再次查看结果如下：
127.0.0.1:8003> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:3
cluster_stats_messages_ping_sent:39
cluster_stats_messages_pong_sent:37
cluster_stats_messages_meet_sent:4
cluster_stats_messages_sent:80
cluster_stats_messages_ping_received:34
cluster_stats_messages_pong_received:43
cluster_stats_messages_meet_received:3
cluster_stats_messages_received:80
以上信息显示集群状态ok。
设置集群密码
上面提供了publicisworldwide/redis-cluster的镜像不能设置永久密码，是因为它没有给redis.conf写入权限，在我们重创建的镜像里是有写入权限的，所以可以创建永久密码。
集群构建完毕后再通过config set + config rewrite命令逐个机器设置密码
在之前的镜像上设置密码会有权限问题，如下：
127.0.0.1:8001> config set masterauth 1111
OK
127.0.0.1:8001> config set requirepass 1111
OK
127.0.0.1:8001> auth
(error) ERR wrong number of arguments for 'auth' command
127.0.0.1:8001> auth 1111
OK
127.0.0.1:8001> config rewrite
(error) ERR Rewriting config file: Permission denied
在新的镜像上添加密码，如下：
[root@localhost src]# ./redis-cli -c -p 8001
127.0.0.1:8001> config set masterauth 1111
OK
127.0.0.1:8001> config set requirepass 1111
OK
127.0.0.1:8001> config rewrite
(error) NOAUTH Authentication required.
127.0.0.1:8001> auth 1111
OK
127.0.0.1:8001> config rewrite
OK
当重登录8001机器并get数据时就会提示需要密码了。其它机器也要一并设置密码才行。
查看密码文件
是否将密码写入到配置文件redis.conf，可以登录到容器查看即可。
docker exec -it 容器ID /bin/bash
cat /usr/local/etc/redis.conf
可以看到masterauth和requirepass被追加到文件的最后,即使重启密码也还生效。
在单机上部署意义不大，可以把slave结点放到不同的服务器上,再通过docker的redis-trib容器来连接,十分灵活和方便。
异常处理
在第三步执行可能会现以下异常，如下：
[root@localhost redis-cluster]# docker run --rm -it inem0o/redis-trib create --replicas 1 172.19.165.222:8001 172.19.165.222:8002 172.19.165.222:8003 172.19.165.222:8004 172.19.165.222:8005 172.19.165.222:8006
>>> Creating cluster
[ERR] Node 172.19.165.222:8001 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
这个异常是因为在目录下/data/redis/800*已经存在redis的配置文件，要先清除掉。
[root@localhost redis-cluster]# rm /data/redis/800* -rf
