Redis --- 最小高可用集群部署

Redis部署最好是6台服务器，但是因为业务量不够，为了节约资源，我们只使用3台服务器，每台服务器上部署两个节点，也可以实现高可用，但是需要时刻监控Redis的主从配置是否合理，保证3个主节点分别在3台服务器上，如果有任意两个主节点分布在同一台服务器，则需要调整。

示例环境

操作系统：CentOS 7.2
服务器：192.168.58.201, 192.168.58.203, 192.168.58.205
端口：6379,6380
版本号：4.0.11

安装Redis服务器

wget http://download.redis.io/releases/redis-4.0.11.tar.gz
tar xzf redis-4.0.11.tar.gz
cd redis-4.0.11
make
sudo make install

部署实例（每台服务器两个实例）

执行脚本./setup.sh，脚本内容如下：

#!/usr/bin/env bash
set -x

## configuration
port1=6379
port2=6380
version=4.0.11

basepath=$(cd `dirname $0`; pwd)

## config two redis
cd ${basepath}
mkdir redis-${port1} redis-${port2}
cp redis-4.0.11/redis.conf ./


sed -i  's/^bind 127.0.0.1$/# bind 127.0.0.1/g' redis.conf
sed -i  's/^daemonize no$/daemonize yes/g' redis.conf
sed -i  "s/^pidfile \/var\/run\/redis_${port1}.pid$/pidfile redis.pid/g" redis.conf

sed -i  's/^# cluster-enabled yes$/cluster-enabled yes/g' redis.conf
sed -i  's/^# cluster-config-file nodes-${port1}.conf$/cluster-config-file nodes.conf/g' redis.conf

sed -i  's/^protected-mode yes$/protected-mode no/g' redis.conf
sed -i  's/^logfile ""$/logfile "redis.log"/g' redis.conf


sed "s/^port 6379$/port ${port1}/g" redis.conf > redis-${port1}/redis.conf
sed "s/^port 6379$/port ${port2}/g" redis.conf > redis-${port2}/redis.conf

sed -i  "s|^dir .\/$|dir ${basepath}/redis-${port1}|g" redis-${port1}/redis.conf
sed -i  "s|^dir .\/$|dir ${basepath}/redis-${port2}|g" redis-${port2}/redis.conf

## clean
rm redis.conf

## start
redis-server redis-${port1}/redis.conf
redis-server redis-${port2}/redis.conf

集群配置

在3台服务器上部署了6个节点后，开始执行集群配置。因为CentOS 7使用yum安装的ruby是2.0.0版本，不符合redis要求，所以使用rvm安装更新的版本.

安装RVM

$ gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
$ \curl -sSL https://get.rvm.io | bash -s stable

  * To start using RVM you need to run `source /home/user/.rvm/scripts/rvm`
    in all your open shell windows, in rare cases you need to reopen all shell windows.
$ source /home/user/.rvm/scripts/rvm

安装ruby

目前使用ruby2.5.1

#安装ruby
rvm install  2.5.1
#使用新版本
rvm use  2.5.1
#查看当前版本
ruby --version

安装gem

tar xzf rubygems-2.7.7.tgz
cd rubygems-2.7.7
sudo ruby setup.rb
# 如果出现错误：sudo: ruby: command not found，则使用下面命令
sudo env "PATH=$PATH" ruby setup.rb

安装redis gem

# gem直接安装
gem install redis

# 或者使用gem离线安装
wget https://rubygems.org/downloads/redis-4.0.2.gem
sudo gem install --force --local redis-4.0.2.gem
# 如果出现错误：sudo: gem: command not found，则使用下面命令
sudo env "PATH=$PATH" gem install --force --local redis-4.0.2.gem

配置

$ redis-4.0.11/src/redis-trib.rb create --replicas 1 192.168.58.201:6379 192.168.58.201:6380 
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.58.201:6379
192.168.58.203:6379
192.168.58.205:6379
Adding replica 192.168.58.203:6380 to 192.168.58.201:6379
Adding replica 192.168.58.205:6380 to 192.168.58.203:6379
Adding replica 192.168.58.201:6380 to 192.168.58.205:6379
M: ae78bc405cfedda5905bd0574ac2250a86cb363c 192.168.58.201:6379
   slots:0-5460 (5461 slots) master
S: ee1b2d1e064dd82bea1adba3bda988fcfa02d613 192.168.58.201:6380
   replicates 3f90145d7e97e9c3c8ce508aebe343f4181ddea3
M: eb289f0ea74a04bc7e743a461fd845e7d6c82549 192.168.58.203:6379
   slots:5461-10922 (5462 slots) master
S: 83fff732d029621d71d3b8e30773d6ce1043d389 192.168.58.203:6380
   replicates ae78bc405cfedda5905bd0574ac2250a86cb363c
M: 3f90145d7e97e9c3c8ce508aebe343f4181ddea3 192.168.58.205:6379
   slots:10923-16383 (5461 slots) master
S: 2b962ca992c64a40e57edde4ac4d51d0ec201f62 192.168.58.205:6380
   replicates eb289f0ea74a04bc7e743a461fd845e7d6c82549
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.....
>>> Performing Cluster Check (using node 192.168.58.201:6379)
M: ae78bc405cfedda5905bd0574ac2250a86cb363c 192.168.58.201:6379
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: eb289f0ea74a04bc7e743a461fd845e7d6c82549 192.168.58.203:6379
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: 83fff732d029621d71d3b8e30773d6ce1043d389 192.168.58.203:6380
   slots: (0 slots) slave
   replicates ae78bc405cfedda5905bd0574ac2250a86cb363c
S: ee1b2d1e064dd82bea1adba3bda988fcfa02d613 192.168.58.201:6380
   slots: (0 slots) slave
   replicates 3f90145d7e97e9c3c8ce508aebe343f4181ddea3
S: 2b962ca992c64a40e57edde4ac4d51d0ec201f62 192.168.58.205:6380
   slots: (0 slots) slave
   replicates eb289f0ea74a04bc7e743a461fd845e7d6c82549
M: 3f90145d7e97e9c3c8ce508aebe343f4181ddea3 192.168.58.205:6379
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.192.168.58.203:6379 192.168.58.203:6380 192.168.58.205:6379 192.168.58.205:6380

常见问题

sudo: ruby: command not found: sudo env "PATH=$PATH" ruby
no such file to load -- zlib

yum -y install zlib-devel
#进入ruby源码文件夹,安装ruby自身提供的zlib包
cd ruby-2.5.1/ext/zlib
ruby ./extconf.rb
make
make install

make: *** No rule to make target /include/ruby.h', needed byzlib.o'

vim Makefile
#zlib.o: $(top_srcdir)/include/ruby.h #把这一行替换成下面一行
zlib.o: ../../include/ruby.h

Redis系列

Redis --- 单实例集群模式

参考

Redis集群模式部署