node节点加入k8s集群失败

2020-09-04

node节点加入k8s集群失败（kubeadm方式部署）

报错1：detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd"

[root@k8s-node2 ~]# kubeadm join 192.168.191.133:6443 --token xvnp3x.pl6i8ikcdoixkaf0 \
>     --discovery-token-ca-cert-hash sha256:9f90161043001c0c75fac7d61590734f844ee507526e948f3647d7b9cfc1362d
[preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`

原因：cgroup和systemd有冲突
解决：

[root@k8s-node2 ~]# docker info | grep Cgroup
WARNING: IPv4 forwarding is disabled
Cgroup Driver: cgroupfs

通过以上命令查到当前的cgroup driver 为cgroupfs，需改为systemd

[root@k8s-node2 ~]# ls /etc/docker/daemon.json        #这个文件是查不到的，但是不用理，直接执行下面的操作
ls: cannot access /etc/docker/daemon.json: No such file or directory

 cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF
[root@k8s-node2 ~]# systemctl daemon-reload
[root@k8s-node2 ~]# systemctl restart docker
[root@k8s-node2 ~]# docker info | grep Cgroup         #再次查看cgroup driver 已改为systemd
Cgroup Driver: systemd
#再次加入集群，成功
[root@k8s-node2 ~]# kubeadm join 192.168.191.133:6443 --token xvnp3x.pl6i8ikcdoixkaf0 \
    --discovery-token-ca-cert-hash sha256:9f90161043001c0c75fac7d61590734f844ee507526e948f3647d7b9cfc1362d

报错2：error execution phase preflight: unable to fetch the kubeadm-config ConfigMap: failed to get config map: Unauthorized

[root@k8s-node2 ~]# kubeadm join 192.168.191.133:6443 --token xvnp3x.pl6i8ikcdoixkaf0 \
>     --discovery-token-ca-cert-hash sha256:9f90161043001c0c75fac7d61590734f844ee507526e948f3647d7b9cfc1362d
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
error execution phase preflight: unable to fetch the kubeadm-config ConfigMap: failed to get config map: Unauthorized

原因：token令牌失效
解决：

#在master上执行
[root@k8s-master ~]# kubeadm token create       #生成新token
56ehzj.hpxea29zdu2w45hf
[root@k8s-master ~]# kubeadm token create --print-join-command         #生成加入集群的命令
kubeadm join 192.168.191.133:6443 --token wagma2.huev9ihugawippas     --discovery-token-ca-cert-hash sha256:9f90161043001c0c75fac7d61590734f844ee507526e948f3647d7b9cfc1362d 
#在node节点上执行
[root@k8s-node2 ~]# kubeadm join 192.168.191.133:6443 --token wagma2.huev9ihugawippas     --discovery-token-ca-cert-hash sha256:9f90161043001c0c75fac7d61590734f844ee507526e948f3647d7b9cfc1362d
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.14" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Activating the kubelet service
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
#成功！

#可在master上查询node节点是否加入成功
[root@k8s-master ~]# kubectl get nodes

报错3：error execution phase kubelet-start: configmaps "kubelet-config-1.19" is forbidden: User "system:bootstrap:xvnp3x" cannot get resource "configmaps" in API group "" in the namespace "kube-system"

[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.19" ConfigMap in the kube-system namespace
error execution phase kubelet-start: configmaps "kubelet-config-1.19" is forbidden: User "system:bootstrap:xvnp3x" cannot get resource "configmaps" in API group "" in the namespace "kube-system"

原因：kubelet版本与master不一致(master上的版本：kubelet-1.14.0)
解决：

yum -y remove kubelet kubeadm kubectl      #卸载当前版本的kube系列
yum -y install kubelet-1.14.0
yum -y install kubectl-1.14.0
yum -y install kubeadm-1.14.0
systemctl start kubelet && systemctl enable kubelet 
kubeadm join 192.168.191.133:6443 --token xvnp3x.pl6i8ikcdoixkaf0 \
    --discovery-token-ca-cert-hash sha256:9f90161043001c0c75fac7d61590734f844ee507526e948f3647d7b9cfc1362d

报错4：/etc/kubernetes/kubelet.conf already exists

[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
    [ERROR FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists
    [ERROR FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`

原因：有残留文件
解决：

rm -rf /etc/kubernetes/kubelet.conf /etc/kubernetes/pki/ca.crt       #删除k8s配置文件和证书文件
kubeadm join 192.168.191.133:6443 --token xvnp3x.pl6i8ikcdoixkaf0 \
    --discovery-token-ca-cert-hash sha256:9f90161043001c0c75fac7d61590734f844ee507526e948f3647d7b9cfc1362d