通过云工作流和计算引擎API在GCE上启动容器时对容器注册表进行身份验证

Question

我大部分时间都试图遵循本指南来部署Google，以便在计算引擎上启动一个容器优化的VM，运行一个长期运行的任务，然后清理VM。我还在使用用Terraform创建工作流中的GCP文档。

我已经建立了自己的形象。映像在本地成功运行，当我通过控制台启动它时，它也在容器优化的VM上运行。该图像构建在云构建上的CI/CD管道中，并上传到。

但是，当我试图通过Cloud控制台启动容器时，我会发现一个与GCR身份验证有关的错误。这是来自云日志记录的堆栈跟踪。

​

我无法解释这一点，因为从控制台启动的VM和通过工作流启动的VM都使用具有相同作用域的相同服务帐户。下面的指南建议使用与已知的工作VM等效的REST--并将其转换为YAML配置，以便在Terraform代码中使用。这就是我所做的。比较一下控制台里的这两台机器。在我看来是一样的。

这是用于创建工作流的Terraform代码。我将提交SHA传递到Terraform脚本以引用容器。我已经确认，构建在云构建上的容器可以在我的本地机器上下载和运行。

resource "google_workflows_workflow" "workflows_example" {
  name            = "scraper-workflow"
  region          = "us-central1"
  description     = "Scraper workflow"
  service_account = "scraper-workflow-executor@rdmops-219503.iam.gserviceaccount.com"
  source_contents = <<-EOF
  # FYI, In terraform you need to escape the $$ or it will cause errors.

  - init:
      assign:
      - commitSHA: ${var.SCRAPER_IMAGE_COMMIT_SHA}
      - projectId: $${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
      - projectNumber: $${sys.get_env("GOOGLE_CLOUD_PROJECT_NUMBER")}
      - zone: "us-central1-a"
      - machineType: "e2-medium"
      - ticker: $${args.ticker}
      - instanceName: $${ticker+"-scraper"}
  - create_and_start_vm:
      call: googleapis.compute.v1.instances.insert
      args:
        project: $${projectId}
        zone: $${zone}
        body:
          canIpForward: false
          confidentialInstanceConfig:
            enableConfidentialCompute: false
          deletionProtection: false
          shieldedInstanceConfig:
            enableIntegrityMonitoring: true
            enableSecureBoot: false
            enableVtpm: true
          tags:
            items:
            - http-server
            - https-server
          name: $${instanceName}
          labels:
            - "container-vm": "cos-stable-97-16919-29-40"
          machineType: $${"zones/" + zone + "/machineTypes/" + machineType}
          disks:
          - initializeParams:
              diskSizeGb: "10"
              diskType: "projects/rdmops-219503/zones/us-central1-a/diskTypes/pd-balanced"
              sourceImage: "projects/cos-cloud/global/images/cos-stable-97-16919-29-40"
            boot: true
            autoDelete: true
            deviceName: $${instanceName}
          # Needed to make sure the VM has an external IP
          networkInterfaces:
          - accessConfigs:
            - name: "External NAT"
              networkTier: "PREMIUM"
            stackType: "IPV4_ONLY"
            subnetwork: "projects/rdmops-219503/regions/us-central1/subnetworks/default"
          # The container to run
          metadata:
              items:
              - key: "google-logging-enabled"
                value: "true"
              - key: "gce-container-declaration"
                value: '$${"spec:\n  containers:\n  - name: scraper-workflow\n    image: gcr.io/" + projectId + "/scraper-workflow:" + commitSHA + "\n    stdin: false\n    tty: false\n  restartPolicy: Never\n"}'

          # Needed to be able to pull down and run the container
          serviceAccounts:
          - email: 937088654099-compute@developer.gserviceaccount.com
            scopes:
            - https://www.googleapis.com/auth/devstorage.read_only
            - https://www.googleapis.com/auth/logging.write
            - https://www.googleapis.com/auth/monitoring.write
            - https://www.googleapis.com/auth/servicecontrol
            - https://www.googleapis.com/auth/service.management.readonly
            - https://www.googleapis.com/auth/trace.append
  - log_wait_for_vm_network:
      call: sys.log
      args:
        data: $${"Waiting for VM network to initialize"}
  - wait_for_vm_network:
      call: sys.sleep
      args:
          seconds: 10
  - get_instance:
      call: googleapis.compute.v1.instances.get
      args:
        instance: $${instanceName}
        project: $${projectId}
        zone: $${zone}
      result: instance
  - extract_external_ip_and_construct_urls:
      assign:
          - external_ip: $${instance.networkInterfaces[0].accessConfigs[0].natIP}
          - base_url: $${"http://" + external_ip + "/"}
          - start_url: $${base_url + "start"}
          - poll_url: $${base_url + "poll"}
  # Redacted rest of workflow for simplicity
EOF
}

这是我通过控制台创建的实例的其余部分。这个很好用。SA在我看来是一样的。还有显微镜..。所有其他的信息..。

POST https://www.googleapis.com/compute/v1/projects/rdmops-219503/zones/us-central1-a/instances
{
  "canIpForward": false,
  "confidentialInstanceConfig": {
    "enableConfidentialCompute": false
  },
  "deletionProtection": false,
  "description": "",
  "disks": [
    {
      "autoDelete": true,
      "boot": true,
      "deviceName": "instance-1",
      "initializeParams": {
        "diskSizeGb": "10",
        "diskType": "projects/rdmops-219503/zones/us-central1-a/diskTypes/pd-balanced",
        "labels": {},
        "sourceImage": "projects/cos-cloud/global/images/cos-stable-97-16919-29-40"
      },
      "mode": "READ_WRITE",
      "type": "PERSISTENT"
    }
  ],
  "displayDevice": {
    "enableDisplay": false
  },
  "guestAccelerators": [],
  "keyRevocationActionType": "NONE",
  "labels": {
    "container-vm": "cos-stable-97-16919-29-40"
  },
  "machineType": "projects/rdmops-219503/zones/us-central1-a/machineTypes/e2-medium",
  "metadata": {
    "items": [
      {
        "key": "ssh-keys",
        "value": "XXXXX"
      },
      {
        "key": "gce-container-declaration",
        "value": "spec:\n  containers:\n  - name: instance-9\n    image: gcr.io/rdmops-219503/workflow-scraper:test3\n    args:\n    - ''\n    stdin: false\n    tty: false\n  restartPolicy: Never\n# This container declaration format is not public API and may change without notice. Please\n# use gcloud command-line tool or Google Cloud Console to run Containers on Google Compute Engine."
      }
    ]
  },
  "name": "instance-9",
  "networkInterfaces": [
    {
      "accessConfigs": [
        {
          "name": "External NAT",
          "networkTier": "PREMIUM"
        }
      ],
      "stackType": "IPV4_ONLY",
      "subnetwork": "projects/rdmops-219503/regions/us-central1/subnetworks/default"
    }
  ],
  "reservationAffinity": {
    "consumeReservationType": "ANY_RESERVATION"
  },
  "scheduling": {
    "automaticRestart": true,
    "onHostMaintenance": "MIGRATE",
    "preemptible": false,
    "provisioningModel": "STANDARD"
  },
  "serviceAccounts": [
    {
      "email": "937088654099-compute@developer.gserviceaccount.com",
      "scopes": [
        "https://www.googleapis.com/auth/devstorage.read_only",
        "https://www.googleapis.com/auth/logging.write",
        "https://www.googleapis.com/auth/monitoring.write",
        "https://www.googleapis.com/auth/servicecontrol",
        "https://www.googleapis.com/auth/service.management.readonly",
        "https://www.googleapis.com/auth/trace.append"
      ]
    }
  ],
  "shieldedInstanceConfig": {
    "enableIntegrityMonitoring": true,
    "enableSecureBoot": false,
    "enableVtpm": true
  },
  "tags": {
    "items": [
      "http-server",
      "https-server"
    ]
  },
  "zone": "projects/rdmops-219503/zones/us-central1-a"
}

Answer 1

问题的原因似乎是找不到您的映像，而不是不允许您的服务帐户访问Container。

与GCE实例关联的服务帐户只需要云存储读写访问来获取私有Docker映像。

在我测试时，权限错误将类似于以下内容：

pull access denied for gcr.io/<PROJECT_ID/IMAGE:TAG>, repository does not exist or may require 'docker login'

但是，当图像找不到时，将返回您正在看到的错误消息。我认为问题的根源在于用于标识映像的提交SHA，因为我可以使用预定义的容器标记执行工作流(类似于在REST for GCE中这样做)。

如果没有看到您的整个CI管道配置，就无法确认它，但它应该是需要重点关注的部分。如果有用的话请告诉我。