GKE上的ImagePullBackOff与私有谷歌云存储库

Question

我正在使用以下(标准)部署在GKE中创建一个部署

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      component: api
  template:
    metadata:
      labels:
        component: api
    spec:
      containers:
      - name: api
        image: eu.gcr.io/xxxx-xxx/api:latest
        imagePullPolicy: Always
        resources:
          requests:
            memory: "320Mi"
            cpu: "100m"
          limits:
            memory: "450Mi"
            cpu: "150m"
        ports:
        - containerPort: 5010

然而，出于某种原因，GKE抱怨权限问题。容器位于同一个项目的容器注册表中，并且是私有的，但据我所知，如果它是GCP项目，GKE应该能够访问。GKE集群是vpc原生的(如果这可能会有所不同)，因为这是我过去使用相同的容器和安装程序运行的集群所能想到的唯一区别。

Events:
  Type     Reason     Age                    From                                                     Message
  ----     ------     ----                   ----                                                     -------
  Normal   Scheduled  34m                    default-scheduler                                        Successfully assigned default/api-deployment-f68977b84-fmhdx to gke-gke-dev-cluster-default-pool-6c6bb127-nw61
  Normal   Pulling    32m (x4 over 33m)      kubelet, gke-gke-dev-cluster-default-pool-6c6bb127-nw61  pulling image "eu.gcr.io/xxxx-xxx/api:latest"
  Warning  Failed     32m (x4 over 33m)      kubelet, gke-gke-dev-cluster-default-pool-6c6bb127-nw61  Failed to pull image "eu.gcr.io/xxxx-xxx/api:latest": rpc error: code = Unknown desc = Error response from daemon: pull access denied for eu.gcr.io/xxxx-xxx/api, repository does not exist or may require 'docker login'
  Warning  Failed     32m (x4 over 33m)      kubelet, gke-gke-dev-cluster-default-pool-6c6bb127-nw61  Error: ErrImagePull
  Normal   BackOff    32m (x6 over 33m)      kubelet, gke-gke-dev-cluster-default-pool-6c6bb127-nw61  Back-off pulling image "eu.gcr.io/xxxx-xxx/api:latest"
  Warning  Failed     3m59s (x131 over 33m)  kubelet, gke-gke-dev-cluster-default-pool-6c6bb127-nw61  Error: ImagePullBackOff

我是否也需要为使用谷歌云存储库的GKE集群添加ImageSecrets，或者是否会有其他问题？

GKE集群是使用GKE的以下gke.tf使用TerraForm创建的

resource "google_container_cluster" "primary" {
  name = "gke-${terraform.workspace}-cluster"
  zone = "${var.region}-b"

  additional_zones = [
    "${var.region}-c",
    "${var.region}-d",
  ]

  # minimum kubernetes version for master
  min_master_version = "${var.min_master_version}"
  # version for the nodes. Should equal min_master_version on create
  node_version       = "${var.node_version}"
  initial_node_count = "${var.gke_num_nodes[terraform.workspace]}"
  network            = "${var.vpc_name}"
  subnetwork         = "${var.subnet_name}"

  addons_config {

    http_load_balancing {
      disabled = false  # this is the default
    }

    horizontal_pod_autoscaling {
      disabled = false
    }

    kubernetes_dashboard {
      disabled = false
    }
  }

  # vpc-native network
  ip_allocation_policy {
#    use_ip_aliases = true
  }

  master_auth {
    username = "${var.gke_master_user}"
    password = "${var.gke_master_pass}"
  }

  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/compute",
      "https://www.googleapis.com/auth/devstorage.read_only",
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]

    labels = {
      env = "${var.gke_label[terraform.workspace]}"
    }

    disk_size_gb = 10
    machine_type = "${var.gke_node_machine_type}"
    tags         = ["gke-node"]
  }
}

运行gcloud gcloud容器集群描述集群给出

nodePools:
- config:
    diskSizeGb: 10
    diskType: pd-standard
    imageType: COS
    labels:
      env: dev
    machineType: n1-standard-1
    metadata:
      disable-legacy-endpoints: 'true'
    oauthScopes:
    - https://www.googleapis.com/auth/monitoring
    - https://www.googleapis.com/auth/devstorage.read_only
    - https://www.googleapis.com/auth/logging.write
    - https://www.googleapis.com/auth/compute
    serviceAccount: default

所以devstorage.read_only似乎就在那里

Answer 1

您的GKE集群节点池是否配置了https://www.googleapis.com/auth/devstorage.read_only OAuth作用域？

要进行检查，可以运行gcloud container clusters describe [CLUSTER NAME]：oauthScopes属性下面列出了作用域。或者在GCP仪表板上检查节点池详细信息：

​

​

应启用Storage。

Answer 2

为了使用GCR，节点需要使用允许从云存储读取的服务帐户和OAuth作用域运行。这里有一些关于这个主题的指导，例如：https://cloud.google.com/kubernetes-engine/docs/how-to/access-scopes#service_account

Answer 3

除了Aleksi注释之外，根据此文档[1]，您还可以使用以下命令检索单个服务帐户的IAM策略：

gcloud iam service-accounts get-iam-policy [SERVICE_ACCOUNT]