GlusterFS部署
准备
1. 目标
安装GlusterFS集群的主要目的是为k8s集群提供分布式持久化存储。
GlusterFS部署使用3台服务器，服务名称与IP如下
/application/ 为单独的分区
/dev/mapper/centos-application 560G 1.3G 558G 1% /application
| name | ip | |
|---|---|---|
| 1 | k8s-server-1 | 192.168.0.162 |
| 2 | k8s-server-2 | 192.168.0.165 |
| 3 | k8s-server-3 | 192.168.0.166 |
2. 环境准备
关闭防火墙
关闭seliux
集群安装
1. 准备repo源
在所有服务器上执行
# cat >/etc/yum.repos.d/gluster-epel.repo << EOF
[gluster]
name=gluster
baseurl=https://buildlogs.centos.org/centos/7/storage/x86_64/gluster-3.8/
gpgcheck=0
enabled=1
EOF
2. 准备数据目录
在所有服务器上执行
mkdir -p /application/data/brick1
ln -s /application/data/ /data
3. 服务安装与启动
yum install glusterfs-server
systemctl start glusterd
systemctl enable glusterd
systemctl status glusterd
4. 集群认证
在k8s-server-1上执行
gluster peer probe k8s-server-2
gluster peer probe k8s-server-3
5. 查看集群集群状态
### 服务器k8s-server-1
# gluster peer status
Number of Peers: 2
Hostname: k8s-server-2
Uuid: 7df14213-28e2-4764-a709-51700b8846ba
State: Peer in Cluster (Connected)
Hostname: k8s-server-3
Uuid: 001879a8-9528-4baa-99cc-50e83d562327
State: Peer in Cluster (Connected)
### 服务器k8s-server-2
# gluster peer status
Number of Peers: 2
Hostname: k8s-server-1
Uuid: bee8fb5e-1297-434f-a0ed-42f9c9e673c7
State: Peer in Cluster (Connected)
Hostname: k8s-server-3
Uuid: 001879a8-9528-4baa-99cc-50e83d562327
State: Peer in Cluster (Connected)
### 服务器k8s-server-3
# gluster peer status
Number of Peers: 2
Hostname: k8s-server-1
Uuid: bee8fb5e-1297-434f-a0ed-42f9c9e673c7
State: Peer in Cluster (Connected)
Hostname: k8s-server-2
Uuid: 7df14213-28e2-4764-a709-51700b8846ba
State: Peer in Cluster (Connected)
6. 创建GlusterFS volume
在所有服务器上执行
mkdir -p /data/brick1/gv0
在k8s-server-1上执行
gluster volume create gv0 replica 3 k8s-server-1:/data/brick1/gv0 k8s-server-2:/data/brick1/gv0 k8s-server-3:/data/brick1/gv0
gluster volume start gv0
查看卷空间
# gluster volume info
Volume Name: gv0
Type: Replicate
Volume ID: e64334b8-61aa-49e9-b310-351bcaf58412
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: k8s-server-1:/data/brick1/gv0
Brick2: k8s-server-2:/data/brick1/gv0
Brick3: k8s-server-3:/data/brick1/gv0
Options Reconfigured:
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on
Gluster 测试
mount -t glusterfs k8s-server-1:/gv0 /mnt
# 在/mnt下创建100个文件
for i in `seq -w 1 100`; do cp -rp /var/log/messages /mnt/copy-test-$i; done
到/data/brick1/gv0 下查看，可以看到文件
