系统部署( CrashLoopBackOff )

Question

我正在尝试将Kubernetes部署到运行Kubernetes v1.14的EKS集群(eks.4)中的kube-system命名空间中。

Kubernetes连接

provider "kubernetes" {
  host                   = var.cluster.endpoint
  token                  = data.aws_eks_cluster_auth.cluster_auth.token
  cluster_ca_certificate = base64decode(var.cluster.certificate)
  load_config_file       = true
}

部署说明(as .tf)

resource "kubernetes_deployment" "kube_state_metrics" {
  metadata {
    name      = "kube-state-metrics"
    namespace = "kube-system"

    labels = {
      k8s-app = "kube-state-metrics"
    }
  }

  spec {
    replicas = 1

    selector {
      match_labels = {
        k8s-app = "kube-state-metrics"
      }
    }

    template {
      metadata {
        labels = {
          k8s-app = "kube-state-metrics"
        }
      }

      spec {
        container {
          name  = "kube-state-metrics"
          image = "quay.io/coreos/kube-state-metrics:v1.7.2"

          port {
            name           = "http-metrics"
            container_port = 8080
          }

          port {
            name           = "telemetry"
            container_port = 8081
          }

          liveness_probe {
            http_get {
              path = "/healthz"
              port = "8080"
            }

            initial_delay_seconds = 5
            timeout_seconds       = 5
          }

          readiness_probe {
            http_get {
              path = "/"
              port = "8080"
            }

            initial_delay_seconds = 5
            timeout_seconds       = 5
          }
        }

        service_account_name = "kube-state-metrics"
      }
    }
  }
}

为了简洁起见，我已经部署了来自https://github.com/kubernetes/kube-state-metrics/tree/master/kubernetes的所有所需的RBAC清单。

当我在上面的部署上运行terraform apply时，Terraform输出如下：kubernetes_deployment.kube_state_metrics: Still creating... [6m50s elapsed]

最后在10米的时候。

下面是kube-state-metrics吊舱日志的输出

I0910 23:41:19.412496       1 main.go:140] metric white-blacklisting: blacklisting the following items:
W0910 23:41:19.412535       1 client_config.go:541] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
W0910 23:41:19.412565       1 client_config.go:546] error creating inClusterConfig, falling back to default config: open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory
F0910 23:41:19.412782       1 main.go:148] Failed to create client: invalid configuration: no configuration has been provided

Answer 1

将以下内容添加到spec中，使我获得了成功的部署。

automount_service_account_token = true

为子孙后代：

resource "kubernetes_deployment" "kube_state_metrics" {
  metadata {
    name      = "kube-state-metrics"
    namespace = "kube-system"

    labels = {
      k8s-app = "kube-state-metrics"
    }
  }

  spec {
    replicas = 1

    selector {
      match_labels = {
        k8s-app = "kube-state-metrics"
      }
    }

    template {
      metadata {
        labels = {
          k8s-app = "kube-state-metrics"
        }
      }

      spec {
        automount_service_account_token = true
        container {
          name  = "kube-state-metrics"
          image = "quay.io/coreos/kube-state-metrics:v1.7.2"

          port {
            name           = "http-metrics"
            container_port = 8080
          }

          port {
            name           = "telemetry"
            container_port = 8081
          }

          liveness_probe {
            http_get {
              path = "/healthz"
              port = "8080"
            }

            initial_delay_seconds = 5
            timeout_seconds       = 5
          }

          readiness_probe {
            http_get {
              path = "/"
              port = "8080"
            }

            initial_delay_seconds = 5
            timeout_seconds       = 5
          }
        }

        service_account_name = "kube-state-metrics"
      }
    }
  }
}

Answer 2

我没试过地形。

我刚刚在本地运行了这个部署，我得到了同样的错误。

请在本地运行您的部署，以查看您的部署和吊舱的状态。

I0910 13:25:49.632847       1 main.go:140] metric white-blacklisting: blacklisting the following items:
W0910 13:25:49.632871       1 client_config.go:541] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.

 and finally:

I0910 13:25:49.634748       1 main.go:185] Testing communication with server
I0910 13:25:49.650994       1 main.go:190] Running with Kubernetes cluster version: v1.12+. git version: v1.12.8-gke.10. git tree state: clean. commit: f53039cc1e5295eed20969a4f10fb6ad99461e37. platform: linux/amd64
I0910 13:25:49.651028       1 main.go:192] Communication with server successful
I0910 13:25:49.651598       1 builder.go:126] Active collectors: certificatesigningrequests,configmaps,cronjobs,daemonsets,deployments,endpoints,horizontalpodautoscalers,ingresses,jobs,limitranges,namespaces,nodes,persistentvolumeclaims,persistentvolumes,poddisruptionbudgets,pods,replicasets,replicationcontrollers,resourcequotas,secrets,services,statefulsets,storageclasses
I0910 13:25:49.651607       1 main.go:226] Starting metrics server: 0.0.0.0:8080
I0910 13:25:49.652149       1 main.go:201] Starting kube-state-metrics self metrics server: 0.0.0.0:8081

核查：

Connected to kube-state-metrics (xx.xx.xx.xx) port 8080 (#0)
 GET /metrics HTTP/1.1
 Host: kube-state-metrics:8080
 User-Agent: curl/7.58.0
 Accept: */*

HTTP/1.1 200 OK
 Content-Type: text/plain; version=0.0.4
 Date: Tue, 10 Sep 2019 13:39:52 GMT
 Transfer-Encoding: chunked

 [49027 bytes data]
 HELP kube_certificatesigningrequest_labels Kubernetes labels converted to 
Prometheus labels.

如果您正在建立自己的形象，请关注千斤顶和文档上的问题

更新:只是为了澄清。

正如我在回答中提到的。我没有尝试使用terraform，但是第一个问题似乎只描述了一个问题，W0910 13:25:49.632871 1 client_config.go:541] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.

因此，我建议在本地运行此部署，并验证日志中的所有错误。后来发生了automount_service_account_token出现问题的情况。这个重要的错误并没有应用到原来的问题上。因此，请按照github上的terraform问题来解决这个问题。

根据github简介

我花了几个小时试图找出为什么服务帐户和部署不能在Terraform中工作，但是在kubectl中没有问题--这是AutomountServiceAccountToken在部署资源中被硬编码为False。

至少应该在资源的Terraform文档中记录这一点，并注意到资源的行为不像kubectl那样。

我希望它能解释这个问题。