Kubernetes通过KubeAdm在OpenStack上使用KVM实例

Question

我已经使用Horizon接口成功地部署了一个“工作的”Kubernetes集群来创建Linux实例：

​

​

根据以下内容配置主机：https://kubernetes.io/docs/setup/independent/high-availability/

我现在可以说我有一个Kubernetes集群：

$ kubectl get nodes
NAME               STATUS    ROLES     AGE       VERSION
kube-apiserver-1   Ready     master    1d        v1.12.2
kube-apiserver-2   Ready     master    1d        v1.12.2
kube-apiserver-3   Ready     master    1d        v1.12.2
kube-node-1        Ready     <none>    21h       v1.12.2
kube-node-2        Ready     <none>    21h       v1.12.2
kube-node-3        Ready     <none>    21h       v1.12.2
kube-node-4        Ready     <none>    21h       v1.12.2

然而，要超越这一点已经被证明是一场相当艰难的斗争。我不能创建可用的服务和核心，这是一个看起来不可用的基本组件：

$ kubectl -n kube-system get pods
NAME                                       READY     STATUS             RESTARTS   AGE
coredns-576cbf47c7-4gdnc                   0/1       CrashLoopBackOff   288        23h
coredns-576cbf47c7-x4h4v                   0/1       CrashLoopBackOff   288        23h
kube-apiserver-kube-apiserver-1            1/1       Running            0          1d
kube-apiserver-kube-apiserver-2            1/1       Running            0          1d
kube-apiserver-kube-apiserver-3            1/1       Running            0          1d
kube-controller-manager-kube-apiserver-1   1/1       Running            3          1d
kube-controller-manager-kube-apiserver-2   1/1       Running            1          1d
kube-controller-manager-kube-apiserver-3   1/1       Running            0          1d
kube-flannel-ds-amd64-2zdtd                1/1       Running            0          20h
kube-flannel-ds-amd64-7l5mr                1/1       Running            0          20h
kube-flannel-ds-amd64-bmvs9                1/1       Running            0          1d
kube-flannel-ds-amd64-cmhkg                1/1       Running            0          1d
...

pod中的错误表明它无法到达kubernetes服务：

$ kubectl -n kube-system logs coredns-576cbf47c7-4gdnc
E1121 18:04:48.928055       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:355: Failed to list *v1.Namespace: Get https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
E1121 18:04:48.928688       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:348: Failed to list *v1.Service: Get https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
E1121 18:04:48.928917       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:350: Failed to list *v1.Endpoints: Get https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
E1121 18:05:19.929869       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:355: Failed to list *v1.Namespace: Get https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
E1121 18:05:19.930819       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:348: Failed to list *v1.Service: Get https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
E1121 18:05:19.931517       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:350: Failed to list *v1.Endpoints: Get https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
E1121 18:05:50.932159       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:355: Failed to list *v1.Namespace: Get https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
E1121 18:05:50.932722       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:348: Failed to list *v1.Service: Get https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
E1121 18:05:50.933179       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:350: Failed to list *v1.Endpoints: Get https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
2018/11/21 18:06:07 [INFO] SIGTERM: Shutting down servers then terminating
E1121 18:06:21.933058       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:355: Failed to list *v1.Namespace: Get https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
E1121 18:06:21.934010       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:348: Failed to list *v1.Service: Get https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
E1121 18:06:21.935107       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:350: Failed to list *v1.Endpoints: Get https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout

$ kubectl -n kube-system描述pod/coredns-576cbf47c7-dk7sh

...
Events:
  Type     Reason     Age                From                  Message
  ----     ------     ----               ----                  -------
  Normal   Scheduled  25m                default-scheduler     Successfully assigned kube-system/coredns-576cbf47c7-dk7sh to kube-node-3
  Normal   Pulling    25m                kubelet, kube-node-3  pulling image "k8s.gcr.io/coredns:1.2.2"
  Normal   Pulled     25m                kubelet, kube-node-3  Successfully pulled image "k8s.gcr.io/coredns:1.2.2"
  Normal   Created    20m (x3 over 25m)  kubelet, kube-node-3  Created container
  Normal   Killing    20m (x2 over 22m)  kubelet, kube-node-3  Killing container with id docker://coredns:Container failed liveness probe.. Container will be killed and recreated.
  Normal   Pulled     20m (x2 over 22m)  kubelet, kube-node-3  Container image "k8s.gcr.io/coredns:1.2.2" already present on machine
  Normal   Started    20m (x3 over 25m)  kubelet, kube-node-3  Started container
  Warning  Unhealthy  4m (x36 over 24m)  kubelet, kube-node-3  Liveness probe failed: HTTP probe failed with statuscode: 503
  Warning  BackOff    17s (x22 over 8m)  kubelet, kube-node-3  Back-off restarting failed container

kubernetes服务已经存在，并且似乎已正确自动配置：

$ kubectl获取服务

NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   23h

$ kubectl描述服务/kubernetes

Name:              kubernetes
Namespace:         default
Labels:            component=apiserver
                   provider=kubernetes
Annotations:       <none>
Selector:          <none>
Type:              ClusterIP
IP:                10.96.0.1
Port:              https  443/TCP
TargetPort:        6443/TCP
Endpoints:         192.168.5.19:6443,192.168.5.24:6443,192.168.5.29:6443
Session Affinity:  None
Events:            <none>

$ kubectl获取端点

NAME         ENDPOINTS                                               AGE
kubernetes   192.168.5.19:6443,192.168.5.24:6443,192.168.5.29:6443   23h

我一直怀疑我在网络层遗漏了一些东西，这个问题与中子有关。关于如何使用其他工具安装Kubernetes以及如何在OpenStack中安装Kubernetes，有很多HOWTO，但我还没有找到一个指南来解释如何通过使用Horizon界面创建KVM并处理安全组和网络问题来安装它。顺便说一句，主机和节点之间的所有IPv4/TCP端口都是开放的。

有没有人有一个指南来解释这个场景？

Answer 1

这里的问题是一个受污染的etcd集群。当我使用以下指令重新构建外部etcd集群并从头开始时：https://kubernetes.io/docs/setup/independent/high-availability/#external-etcd，所有项目都按预期工作。似乎没有可用于重置法兰绒pod网络的etcd条目的工具。