weave-net pods处于挂起状态，调度程序日志中出现错误

Question

我正在尝试建立一个新的kubernetes集群，并且面临着使用weave作为网络解决方案的问题。编织pods挂起在挂起状态，并且kubectl命令行中没有可用的事件/日志。

我正在尝试从头开始建立一个kubernetes集群，作为在线课程的一部分。我已经设置了主节点- api服务器、控制器管理器和调度器都已启动并运行。以及运行kubelets和kube-proxy的工作节点。

节点状态：

vagrant@master-1:~$ kubectl get nodes -n kube-system

NAME STATUS ROLES AGE VERSION worker-1 NotReady <none> 25h v1.13.0 worker-2 NotReady <none> 9h v1.13.0

作为启用网络的下一步，我将使用weave。我已经在worker节点上安装了weave和解压缩。

现在，当我尝试运行下面的命令时：

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

我看到DaemonSet正在初始化，但创建的pods继续处于“挂起状态”。

vagrant@master-1:~$ kubectl get pods -n kube-system

NAME READY STATUS RESTARTS AGE weave-net-ccrqs 0/2 Pending 0 73m weave-net-vrm5f 0/2 Pending 0 73m

下面的命令：vagrant@master-1:~$ kubectl describe pods -n kube-system不返回任何正在进行的事件。

从调度程序服务日志中，我可以看到记录了以下错误。

Oct 13 16:46:51 master-2 kube-scheduler[14569]: E1013 16:46:51.973883   14569 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:anonymous" cannot list resource "statefulsets" in API group "apps" at the cluster scope
Oct 13 16:46:51 master-2 kube-scheduler[14569]: E1013 16:46:51.982228   14569 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
Oct 13 16:46:52 master-2 kube-scheduler[14569]: E1013 16:46:52.338171   14569 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:anonymous" cannot list resource "persistentvolumes" in API group "" at the cluster scope
Oct 13 16:46:52 master-2 kube-scheduler[14569]: E1013 16:46:52.745288   14569 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Oct 13 16:46:52 master-2 kube-scheduler[14569]: E1013 16:46:52.765103   14569 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:anonymous" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
Oct 13 16:46:52 master-2 kube-scheduler[14569]: E1013 16:46:52.781419   14569 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:anonymous" cannot list resource "replicasets" in API group "apps" at the cluster scope
Oct 13 16:46:52 master-2 kube-scheduler[14569]: E1013 16:46:52.785872   14569 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:anonymous" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
Oct 13 16:46:52 master-2 kube-scheduler[14569]: E1013 16:46:52.786117   14569 reflector.go:134] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:232: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Oct 13 16:46:52 master-2 kube-scheduler[14569]: E1013 16:46:52.786790   14569 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Node: nodes is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Oct 13 16:46:52 master-2 kube-scheduler[14569]: E1013 16:46:52.787016   14569 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:anonymous" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

由于我是kubernetes的新手，如果我错过了添加相关信息，请原谅。将立即生效。需要好心的帮助。

调度器新增kubeconfig：

    {
      kubectl config set-cluster kubernetes-the-hard-way \
        --certificate-authority=ca.crt \
        --embed-certs=true \
        --server=https://127.0.0.1:6443 \
        --kubeconfig=kube-scheduler.kubeconfig

      kubectl config set-credentials system:kube-scheduler \
        --client-certificate=kube-scheduler.crt \
        --client-key=kube-scheduler.key \
        --embed-certs=true \
        --kubeconfig=kube-scheduler.kubeconfig

      kubectl config set-context default \
        --cluster=kubernetes-the-hard-way \
        --user=system:kube-scheduler \
        --kubeconfig=kube-scheduler.kubeconfig

      kubectl config use-context default --kubeconfig=kube- 
   scheduler.kubeconfig
    }

添加调度器服务定义：

cat <<EOF | sudo tee /etc/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes

[Service]
ExecStart=/usr/local/bin/kube-scheduler \\
  --kubeconfig=/var/lib/kubernetes/kube-scheduler.kubeconfig \\
  --address=127.0.0.1 \\
  --leader-elect=true \\
  --v=2
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

已使用以下命令启动计划程序：

sudo systemctl enable kube-scheduler
sudo systemctl start kube-scheduler

组件状态：

vagrant@master-1:~$ kubectl get componentstatuses --kubeconfig admin.kubeconfig
NAME                 STATUS    MESSAGE             ERROR
scheduler            Healthy   ok
controller-manager   Healthy   ok
etcd-0               Healthy   {"health":"true"}
etcd-1               Healthy   {"health":"true"}

Answer 1

我已经在参与HA的两个主节点上重新启动了kube调度器和控制器管理器，我相信这允许api服务器的负载均衡器URL生效，并且消除了前面观察到的错误。

在此之后，我设置了一个工作节点并安装了weave，部署了pod并准备好了节点。

vagrant@master-1:~$ kubectl get pods -n kube-system
NAME              READY   STATUS    RESTARTS   AGE
weave-net-zswht   1/2     Running   0          41s
vagrant@master-1:~$ kubectl get nodes
NAME       STATUS   ROLES    AGE     VERSION
worker-1   Ready    <none>   4m51s   v1.13.0

Answer 2

来自调度程序的日志消息表明，它没有配置为在系统帐户下运行-它需要查看正在发生的事情和进行更改的权限。

假设您应该在前面的步骤中对其进行配置。

我不认为这与Weave Net有任何关系--你可能会遇到同样的困难来尝试运行任何东西。

Answer 3

在我的示例中，我未能完全禁用工作节点上的交换，因此在重新启动后，kubelet没有完全初始化，从而导致了类似上面的错误。在使用'swapoff -a‘并删除了有问题的fstab交换行之后，一切正常，kubelet切换到了活动状态。