从kubelet获得证书后，kubenetes无法获取节点信息

Question

我以手动方式安装kubernetes。在master批准kubelet的证书请求后，我无法获取节点信息。

首先，etcd kube-apiserver kube-controller-manager和kube-scheduler已正确安装。Execute commands kubectl get cs得到以下输出：

[root@master cfg]# kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok
scheduler            Healthy   ok
etcd-2               Healthy   {"health":"true"}
etcd-0               Healthy   {"health":"true"}
etcd-1               Healthy   {"health":"true"}

然后，我尝试在主节点和工作节点上安装kubelet。在主节点上执行命令kubectl approve node-csr-V_FXPiKHAtqS_9GH27RCk6hPNWE0nF8bLSH6Ot7C360和kubectl get csr

[root@master cfg]# kubectl get csr
NAME                                                   AGE    SIGNERNAME                                    REQUESTOR           CONDITION
node-csr-Tbrw3ia-c0D7kgIURIe_JnP1PCEdwf9XeATP0KcsyBg   111m   kubernetes.io/kube-apiserver-client-kubelet   kubelet-bootstrap   Approved,Issued
node-csr-V_FXPiKHAtqS_9GH27RCk6hPNWE0nF8bLSH6Ot7C360   131m   kubernetes.io/kube-apiserver-client-kubelet   kubelet-bootstrap   Approved,Issued

但是，我不能从这个集群中获取节点：

[root@master cfg]# kubectl get node
No resources found in default namespace.

任何人都可以拉我一把吗？期待您的帮助或一些想法来追踪问题。谢谢！

Answer 1

我已经找到了这个问题的原因。主节点和工作节点上的kubelet服务没有真正启动。我在/var/log/message中遇到错误，请执行以下操作：

Dec 11 21:21:20 worker1 systemd: Started Kubernetes Kubelet.
Dec 11 21:21:20 worker1 systemd: Starting Kubernetes Kubelet...
Dec 11 21:21:20 worker1 systemd: Started Kubernetes systemd probe.
Dec 11 21:21:20 worker1 systemd: Starting Kubernetes systemd probe.
Dec 11 21:21:20 worker1 kubelet: F1211 21:21:20.905947    3066 server.go:274] failed to run Kubelet: misconfiguration: kubelet cgroup driver: "cgroupfs" is different from docker cgroup driver: "systemd"

这意味着docker的cgroup driver配置与kubelet冲突。更新docker服务文件/usr/lib/systemd/system/docker.service后，设置--exec-opt native.cgroupdriver=cgroupfs：

[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target
Wants=docker-storage-setup.service
Requires=docker-cleanup.timer

[Service]
Type=notify
NotifyAccess=main
EnvironmentFile=-/run/containers/registries.conf
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
Environment=GOTRACEBACK=crash
Environment=DOCKER_HTTP_HOST_COMPAT=1
Environment=PATH=/usr/libexec/docker:/usr/bin:/usr/sbin
ExecStart=/usr/bin/dockerd-current \
          --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current \
          --default-runtime=docker-runc \
          --exec-opt native.cgroupdriver=cgroupfs \
          --userland-proxy-path=/usr/libexec/docker/docker-proxy-current \
          --init-path=/usr/libexec/docker/docker-init-current \
          --seccomp-profile=/etc/docker/seccomp.json \
          $OPTIONS \
          $DOCKER_STORAGE_OPTIONS \
          $DOCKER_NETWORK_OPTIONS \
          $ADD_REGISTRY \
          $BLOCK_REGISTRY \
          $INSECURE_REGISTRY \
          $REGISTRIES
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
TimeoutStartSec=0
Restart=on-abnormal
KillMode=process

[Install]
WantedBy=multi-user.target

然后，重启docker和kubelet服务，问题就解决了。Service kubelet工作正常。

执行命令kubelet get nodes，输出如下：

[root@master ~]# kubectl get nodes
NAME      STATUS     ROLES    AGE    VERSION
master    NotReady   <none>   8s     v1.18.3
worker1   NotReady   <none>   112s   v1.18.3