K3s -拨号TCP10.43.0.1:443:连接:连接被拒绝

Question

我创建了一个K3s多主嵌入式集群，如下所示：

主机名: k3s01

curl -sfL https://get.k3s.io | K3S_TOKEN=xxx INSTALL_K3S_EXEC="server --disable servicelb --disable traefik --bind-address=10.0.0.4 --tls-san 10.0.0.4 --node-external-ip=168.119.x.x --node-ip=10.0.0.4 --flannel-iface=enp7s0 --advertise-address=PUBIP-OF-LB --cluster-init" sh -

主机名: k8s02

curl -sfL https://get.k3s.io | K3S_TOKEN=xxx INSTALL_K3S_EXEC="server --disable servicelb --disable traefik --bind-address=10.0.0.2 --tls-san 10.0.0.2 --node-ip 10.0.0.2 --node-external-ip=168.119.x.x  --flannel-iface=enp7s0 --server=https://10.0.0.4:6443" sh -

主机名: k8s03

curl -sfL https://get.k3s.io | K3S_TOKEN=xxx INSTALL_K3S_EXEC="server --disable servicelb --disable traefik --bind-address=10.0.0.3 --tls-san 10.0.0.3 --node-ip 10.0.0.3 --node-external-ip=168.119.x.x  --flannel-iface=enp7s0 --server=https://10.0.0.4:6443" sh -

我可以通过LB-IP从我的本地机器与kubectl连接！LB: tcp 6443 -> 6443

我也可以在上面的任何节点中使用kubectl。我为Hetzner部署了CSI，这也很好。测试了他们的测试部署！

然而，在这一切之后(到目前为止工作得很好)，我试着安装了。部署开始时没有任何问题。但是，我发现在集群中与apiserver通信存在问题，如入口-nginx控制器的以下日志所示：

E1204 11:42:25.216392       8 leaderelection.go:321] error retrieving resource lock ingress-nginx/ingress-controller-leader-nginx: Get "https://10.43.0.1:443/api/v1/namespaces/ingress-nginx/configmaps/ingress-controller-leader-nginx": dial tcp 10.43.0.1:443: connect: connection refused

嗯，奇怪！好吧，让我们检查一下：

kubectl get svc kubernetes -o yaml

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2020-12-04T11:22:25Z"
  labels:
    component: apiserver
    provider: kubernetes
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .: {}
          f:component: {}
          f:provider: {}
      f:spec:
        f:clusterIP: {}
        f:ports:
          .: {}
          k:{"port":443,"protocol":"TCP"}:
            .: {}
            f:name: {}
            f:port: {}
            f:protocol: {}
            f:targetPort: {}
        f:sessionAffinity: {}
        f:type: {}
    manager: k3s
    operation: Update
    time: "2020-12-04T11:22:25Z"
  name: kubernetes
  namespace: default
  resourceVersion: "10434"
  selfLink: /api/v1/namespaces/default/services/kubernetes
  uid: f0993556-3b7f-40aa-a293-45170cb03002
spec:
  clusterIP: 10.43.0.1
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 6443
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

看上去像gtm。

kubectl get endpoints -o yaml

apiVersion: v1
items:
- apiVersion: v1
  kind: Endpoints
  metadata:
    creationTimestamp: "2020-12-04T11:22:25Z"
    labels:
      endpointslice.kubernetes.io/skip-mirror: "true"
    managedFields:
    - apiVersion: v1
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:labels:
            .: {}
            f:endpointslice.kubernetes.io/skip-mirror: {}
        f:subsets: {}
      manager: k3s
      operation: Update
      time: "2020-12-04T11:23:39Z"
    name: kubernetes
    namespace: default
    resourceVersion: "808"
    selfLink: /api/v1/namespaces/default/endpoints/kubernetes
    uid: cb450392-b4c9-4c2f-bfde-1a3b20ac4b5d
  subsets:
  - addresses:
    - ip: 167.233.x.x
    - ip: 168.119.x.x
    - ip: 168.119.x.x
    ports:
    - name: https
      port: 6443
      protocol: TCP
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

好吧，为什么酒吧的IPs在这里？让我们从一个舱内检查它，直接调用其中一个IP：

kubectl exec -it ingress-controler-pod-xxxx -- bash

bash-5.0$ curl https://167.233.x.x:6443 --insecure
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "Unauthorized",
  "reason": "Unauthorized",
  "code": 401
}bash-5.0$ curl https://10.43.0.1:443
curl: (7) Failed to connect to 10.43.0.1 port 443: Connection refused

好的..。真奇怪！

有时还会出现一些错误，如：

Error from server: error dialing backend: dial tcp: lookup k8s02: Try again

当我尝试进入一个吊舱或显示日志时出现。只有当我试图对运行在另一个主机上的目标舱执行此操作时，才会发生这种情况。

DNS有什么问题吗？

cat /etc/resolv.conf
nameserver 127.0.0.53
options edns0 trust-ad

我不能用他们的名字来分辨我的主人。但是，我刚刚在K3s设置中指定了in。我需要在主机之间运行DNS吗？我的K3s安装参数有什么问题吗？

Answer 1

我也遇到了一个类似的问题，这是由配置错误的DNS解析引起的，请检查是否可以将节点主机名彼此解析。