概述
kubernetes集群经常会遇到添加新节点master失败的问题，下面了解一下通用的解决方案。
etcd架构
etcd知识图谱
错误现象
安装三个节点master的高可用kubernetes集群，可能会遇到添加新节点master失败的问题。
因etcd失败而导致添加master失败，可按以下方法解决问题：
错误一：
error execution phase check-etcd: etcd cluster is not healthy: context deadline exceeded
错误二：
error execution phase check-etcd: error syncing endpoints with etc: dial tcp 192.168.1.10:2379: connect: connection refused
解决方法
进入etcd集群，删除异常的etcd集群节点，到报异常的节点执行kubeadm reset命令后继 续执行添加master的命令
# kubectl exec -it etcd-192.168.1.10 sh -n kube-system /
# export ETCDCTL_API=3 /
# alias etcdctl='etcdctl --endpoints=https://192.168.1.10:2379 -- cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key'
# etcdctl member list 48d035e72d5a2c65, started, 192.168.1.10, https://192.168.1.11:2380, https://192.168.1.12:2379 9d8522a49fdf6359, started, 192.168.1.10, https://192.168.1.11:2380, https://192.168.1.12:2379 ca3435917a38658e, unstarted, 192.168.1.10, https://192.168.1.11:2380, https://192.168.1.12:2379
# etcdctl member remove ca3435917a38658e Member ca3435917a38658e removed from cluster 8650138bc047cb5
# etcdctl member list 48d035e72d5a2c65, started, 192.168.1.10, https://192.168.1.11:2380, https://192.168.1.12:2379 9d8522a49fdf6359, started, 192.168.1.10, https://192.168.1.11:2380, https://192.168.1.12:2379
扩展实操
遇到状态为Terminating的POD，强制删除POD的方法
kubectl delete po etcd-192.168.1.12 -n kube-system --force kubectl delete po kube-apiserver-192.168.1.12 -n kube-system --force
扩展实操：常用kubectl工具命令
kubectl get po -o wide -A # 查看所有命名空间下POD
kubectl get po -o wide -n kube-system # 查看命名空间kube-system下POD
kubectl logs -f --tail=100 PODID
kubectl describe po PODID # 查看pod属性信息
kubectl get deploy -n kube-system
kubectl get daemonset -n kube-system
