Kubernetes仪表板节点不工作

Question

我正在尝试使用NodePort运行kubernetes仪表板。但这似乎不起作用，我遵循以下步骤：

1. 通过kubeadm版本1.15.3安装kubernetes
2. 安装法兰绒作为吊舱网络
3. 安装以端口类型为NodePort的kubernetes仪表板

当我做kubctl logs ... kubernetes-dashboard时，我得到以下信息

2019/09/03 01:22:31 Starting overwatch
2019/09/03 01:22:31 Using namespace: kubernetes-dashboard
2019/09/03 01:22:31 Using in-cluster config to connect to apiserver
2019/09/03 01:22:31 Using secret token for csrf signing
2019/09/03 01:22:31 Initializing csrf token from kubernetes-dashboard-csrf secret
2019/09/03 01:22:31 Empty token. Generating and storing in a secret kubernetes-dashboard-csrf
2019/09/03 01:22:31 Successful initial request to the apiserver, version: v1.15.3
2019/09/03 01:22:31 Generating JWE encryption key
2019/09/03 01:22:31 New synchronizer has been registered: kubernetes-dashboard-key-holder-kubernetes-dashboard. Starting
2019/09/03 01:22:31 Starting secret synchronizer for kubernetes-dashboard-key-holder in namespace kubernetes-dashboard
2019/09/03 01:22:32 Initializing JWE encryption key from synchronized object
2019/09/03 01:22:32 Creating in-cluster Sidecar client
2019/09/03 01:22:32 Auto-generating certificates
2019/09/03 01:22:32 Successfully created certificates
2019/09/03 01:22:32 Serving securely on HTTPS port: 8443
2019/09/03 01:22:32 Successful request to sidecar

到目前为止，我觉得不错。现在，为了获得nodeport，我做了kubectl get svc --namespace kubernetes-dashboard

现在我得到了

NAME                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
dashboard-metrics-scraper   ClusterIP   10.108.213.154   <none>        8000/TCP        11m
kubernetes-dashboard        NodePort    10.109.111.248   <none>        443:30206/TCP   11m

现在，当我访问https://master-ip:30206时，我只需要超时，日志就会保持不变。

服务yaml：

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard

部署yaml：

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.0.0-beta4
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
          args:
            - --auto-generate-certificates
            - --namespace=kubernetes-dashboard
            # Uncomment the following line to manually specify Kubernetes API server Host
            # If not specified, Dashboard will attempt to auto discover the API server and connect
            # to it. Uncomment only if the default does not work.
            # - --apiserver-host=http://my-address:port
          volumeMounts:
            - name: kubernetes-dashboard-certs
              mountPath: /certs
              # Create on-disk volume to store exec logs
            - mountPath: /tmp
              name: tmp-volume
          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /
              port: 8443
            initialDelaySeconds: 30
            timeoutSeconds: 30
      volumes:
        - name: kubernetes-dashboard-certs
          secret:
            secretName: kubernetes-dashboard-certs
        - name: tmp-volume
          emptyDir: {}
      serviceAccountName: kubernetes-dashboard
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule

应用yaml时输出：

:namespace/kubernetes-dashboard created
serviceaccount/kubernetes-dashboard created
service/kubernetes-dashboard created
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
secret/kubernetes-dashboard-key-holder created
configmap/kubernetes-dashboard-settings created
role.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
deployment.apps/kubernetes-dashboard created
service/dashboard-metrics-scraper created
deployment.apps/dashboard-metrics-scraper created

Answer 1

我终于解决了这个问题，我采取了以下步骤：

我的iptable似乎搞砸了，我在主服务器上执行以下命令时发现了这一点：

curl {master-ip/e.g 192.168.1.11}:{Nodeport of kubernetes dashboard}

这个命令正常工作，我得到了仪表板，同样的卷曲也让我在客户端机器上超时。

因此，我通过在主节点上执行以下命令来修复这个问题：

sudo iptables -A FORWARD -j ACCEPT

现在出现了另一个问题，当您访问时，将产生无效的证书错误。

要解决这一问题，需要执行以下操作：

创建一个名为san.cnf的文件，其内容如下：

[ req ]
default_bits        = 2048
distinguished_name  = req_distinguished_name
req_extensions      = req_ext
prompt              = no
[ req_distinguished_name ]
countryName             = NL
stateOrProvinceName     = Noord Brabant
localityName            = Rosmalen
organizationName        = ARRServices
commonName              = k8s.dashboard.prod
[ req_ext ]
subjectAltName = @alt_names
[alt_names]
DNS.1   = {optional only do this if you reverse proxy}
IP.1    = {IP of the machine that is running the dashboard usually the master node}

# Create certificate by executing the following commands
sudo mkdir /certs    
sudo chmod 777 -R /certs
openssl req -out /certs/dashboard.csr -newkey rsa:2048 -nodes -keyout /certs/dashboard.key -config san.cnf
openssl x509 -req -sha256 -days 3650 -in /certs/dashboard.csr -signkey /certs/dashboard.key -out /certs/dashboard.crt -extensions req_ext -extfile san.cnf
sudo chmod 777 -R /certs

现在，您的kubernetes-dashboard.yml文件应该如下所示：

apiVersion: v1
kind: Namespace
metadata:
  name: kubernetes-dashboard

---

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 31000
  selector:
    k8s-app: kubernetes-dashboard

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard
type: Opaque

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-csrf
  namespace: kubernetes-dashboard
type: Opaque
data:
  csrf: ""

---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-key-holder
  namespace: kubernetes-dashboard
type: Opaque

---

kind: ConfigMap
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-settings
  namespace: kubernetes-dashboard

---

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
rules:
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
  - apiGroups: [""]
    resources: ["secrets"]
    resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
    verbs: ["get", "update", "delete"]
    # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["kubernetes-dashboard-settings"]
    verbs: ["get", "update"]
    # Allow Dashboard to get metrics.
  - apiGroups: [""]
    resources: ["services"]
    resourceNames: ["heapster", "dashboard-metrics-scraper"]
    verbs: ["proxy"]
  - apiGroups: [""]
    resources: ["services/proxy"]
    resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
    verbs: ["get"]

---

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
rules:
  # Allow Metrics Scraper to get metrics from the Metrics server
  - apiGroups: ["metrics.k8s.io"]
    resources: ["pods", "nodes"]
    verbs: ["get", "list", "watch"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubernetes-dashboard
subjects:
  - kind: ServiceAccount
    name: kubernetes-dashboard
    namespace: kubernetes-dashboard

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      containers:
        - name: kubernetes-dashboard
          image: kubernetesui/dashboard:v2.0.0-beta4
          imagePullPolicy: Always
          ports:
            - containerPort: 8443
              protocol: TCP
          args:
            - --auto-generate-certificates=false
            - --namespace=kubernetes-dashboard
            - --tls-cert-file=dashboard.crt
            - --tls-key-file=dashboard.key
            # Uncomment the following line to manually specify Kubernetes API server Host
            # If not specified, Dashboard will attempt to auto discover the API server and connect
            # to it. Uncomment only if the default does not work.
            # - --apiserver-host=http://my-address:port
          volumeMounts:
            - name: kubernetes-dashboard-certs
              mountPath: /certs
              # Create on-disk volume to store exec logs
            - mountPath: /tmp
              name: tmp-volume
          livenessProbe:
            httpGet:
              scheme: HTTPS
              path: /
              port: 8443
            initialDelaySeconds: 30
            timeoutSeconds: 30
      volumes:
        - name: kubernetes-dashboard-certs
          hostPath:
            path: /certs
          #secret:
          #  secretName: kubernetes-dashboard-certs
        - name: tmp-volume
          emptyDir: {}
      serviceAccountName: kubernetes-dashboard
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule

---

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 8000
      targetPort: 8000
  selector:
    k8s-app: dashboard-metrics-scraper

---

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    k8s-app: dashboard-metrics-scraper
  name: dashboard-metrics-scraper
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: dashboard-metrics-scraper
  template:
    metadata:
      labels:
        k8s-app: dashboard-metrics-scraper
    spec:
      containers:
        - name: dashboard-metrics-scraper
          image: kubernetesui/metrics-scraper:v1.0.1
          ports:
            - containerPort: 8000
              protocol: TCP
          livenessProbe:
            httpGet:
              scheme: HTTP
              path: /
              port: 8000
            initialDelaySeconds: 30
            timeoutSeconds: 30
          volumeMounts:
          - mountPath: /tmp
            name: tmp-volume
      serviceAccountName: kubernetes-dashboard
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      volumes:
        - name: tmp-volume
          emptyDir: {}

我所做的更改如下:kubernetes的服务-仪表板将有：

NodePort而不是ClusterIP

• 静态nodePort 31000而不是动态端口
• kubernetes的Args -仪表板部署如下：
• -自动生成证书=false--命名空间=kubernetes-仪表板--tls-cert-file=dashboard.crt --tls-key-file=dashboard.key
• 用hostPath: path替换卷kubernetes-仪表板-证书中的hostPath: /certs #引用生成证书的路径
• 名称:kubernetes-仪表板-certs hostPath: path: /certs

现在执行以下命令：'kubectl应用-f kubernetes-dashboard.yml‘

现在您可以通过{ip}:{nodePort}访问它了。

• ip通常是主节点ip。
• 可以通过以下操作检索nodeport： kubectl获取svc -命名空间kubernetes-仪表板

大多数浏览器都会出现一个错误，这个错误看起来很难看，要解决这个问题，请执行以下操作( Most，使用chrome)：

• 打开铬，进入仪表板的ip和端口，例如"192.168.1.11:31000“
• 您将看到证书不受信任，因此请下载证书:转到developer tools => security =>视图证书，并将证书图标/徽标拖到桌面上。
• 打开密钥链=>转到密钥链"System“=>转到”证书“类别
• 将证书拖到证书列表中。
• 双击证书打开它，单击‘信任’-选择在“时”使用
• 此证书“始终信任”选项。

现在您应该能够访问仪表板了，下一步是创建管理用户，可以这样做：

创建一个名为serviceaccount.yml的文件，其内容如下

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard

创建一个名为rbac.yml的文件，其内容如下：

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

执行下列命令

kubectl apply -f serviceaccount.yml
kubectl apply -f rbac.yml

通过执行以下命令获取令牌

kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep admin-user | awk '{print $1}')

这与kubernetes 1.15.3，对接者18.09，kubernetes-仪表板2.0.0beta 4一起工作。

测试: raspberry pi 4作为主节点，但也应该与其他设备一起工作。

Answer 2

您为kubernetes-仪表板服务帐户创建了ClusterRole，ClusterRoleBinding资源吗？如果没有，可以使用下面的定义文件应用更改吗？

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/05_dashboard-rbac.yaml