日期: 2020 年 3 月 25 日

在K8S里使用NFS做storageclass

参考github:https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client


1.使用helm安装nfs-client,注意填写nfs服务器的地址和暴露路径

$ helm install stable/nfs-client-provisioner --set nfs.server=x.x.x.x --set nfs.path=/exported/path

2.编写storageclass.yaml并使其生效

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: fuseim.pri/ifs

3.设置这个storageclass为默认storageclass

kubectl patch storageclass nfs-client -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

4.注意这边下载quay.azk8s.cn/external_storage/nfs-client-provisioner:v3.1.0-k8s1.11这个镜像可能很慢,可以调整成国内源下载,再重命名。例如下面的命令

docker pull quay.azk8s.cn/external_storage/nfs-client-provisioner:v3.1.0-k8s1.11
docker tag quay.azk8s.cn/external_storage/nfs-client-provisioner:v3.1.0-k8s1.11 quay.io/external_storage/nfs-client-provisioner:v3.1.0-k8s1.11

K8S重新加入master节点,避免etcd错误

我们有时候会有删除,再重新加入master节点的需求,比如master机器改名。这里注意重新加入时,经常会出现etcd报错,如下

[check-etcd] Checking that the etcd cluster is healthy
error execution phase check-etcd: etcd cluster is not healthy: failed to dial endpoint https://192.168.0.92:2379 with maintenance client: context deadline exceeded

这个时候,就需要去还没有停止的master节点里的etcd的pod里去,删除该老master节点对应的etcd信息。


kubernetes节点信息

master03是我们将要删除重新加入的节点

# root@master01:~# kubectl get nodes
NAME       STATUS   ROLES    AGE   VERSION
master01   Ready    master   64d   v1.17.1
master02   Ready    master   95s   v1.17.1
master03   Ready    master   18h   v1.17.1
slaver01   Ready    <none>   64d   v1.17.1
slaver04   Ready    <none>   13d   v1.17.1
slaver05   Ready    <none>   13d   v1.17.1

删除master

在master01机器上执行

kubectl drain master03
kubectl delete node master03

在master03机器上执行

kubeadm reset
rm -rf /etc/kubernetes/manifests/

删除etcd信息

在master01节点上执行命令,进入etcd的容器里

kubectl exec -it etcd-master01 sh

输入命令

etcdctl --endpoints 127.0.0.1:2379 --cacert /etc/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/server.crt --key /etc/kubernetes/pki/etcd/server.key member list

检查返回值

7d39fc3ab8790afc, started, master03, https://192.168.0.93:2380, https://192.168.0.93:2379, false
b54177b91845ab93, started, master01, https://192.168.0.91:2380, https://192.168.0.91:2379, false
bc771924f2f5445f, started, master02, https://192.168.0.92:2380, https://192.168.0.92:2379, false

因为我们的master03机器对应的hash是7d39fc3ab8790afc。我们下一步就是根据hash删除etcd信息,执行如下命令

etcdctl --endpoints 127.0.0.1:2379 --cacert /etc/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/server.crt --key /etc/kubernetes/pki/etcd/server.key member remove 12637f5ec2bd02b8

获取添加master的命令


在master01上输入命令

kubeadm init phase upload-certs --upload-certs

返回34f76df3029230ca3136f5ff689ed54b1af6501a59fb0ea728ff8fed31ad52b4
再在master01上输入命令

kubeadm token create --print-join-command

返回 kubeadm join cluster.kube.com:16443 –token f4amr0.c2nc87swc7jbybut –discovery-token-ca-cert-hash sha256:2c45bcc43dad9cf43c3b7e610c0cdb7d588213d4258fc060e7384276e664922e
通过组合上面的“蓝色字体部分“+“–control-plane –certificate-key“ +“红色字体部分”,获得加入master的完整命令
kubeadm join cluster.kube.com:16443 –token uerys4.h8z3lfo2j3zf8g2u –discovery-token-ca-cert-hash sha256:2c45bcc43dad9cf43c3b7e610c0cdb7d588213d4258fc060e7384276e664922e –control-plane –certificate-key 34f76df3029230ca3136f5ff689ed54b1af6501a59fb0ea728ff8fed31ad52b4


添加Master节点

执行命令

kubeadm join cluster.kube.com:16443 --token uerys4.h8z3lfo2j3zf8g2u --discovery-token-ca-cert-hash sha256:2c45bcc43dad9cf43c3b7e610c0cdb7d588213d4258fc060e7384276e664922e --control-plane --certificate-key 34f76df3029230ca3136f5ff689ed54b1af6501a59fb0ea728ff8fed31ad52b4

苏ICP备18047533号-1