GCP人工智能平台-管道-集群-没有最低可用性。

Question

我不能制造管道。我甚至不能在AI平台管道仪表板上加载示例/教程，因为它似乎无法代理它需要的任何东西。

An error occurred
Error occured while trying to proxy to: ... 

我查看了集群的详细信息，发现了三个有错误的组件：

Deployment  metadata-grpc-deployment     Does not have minimum availability 
Deployment  ml-pipeline  Does not have minimum availability 
Deployment  ml-pipeline-persistenceagent     Does not have minimum availability 

创建集群涉及到大约。3点击在引擎，所以我不认为我搞砸了这一步。

有人知道如何实现“最低可用性”吗？

更新1

节点有足够的资源并且已经准备好了。YAML文件看起来不错。我在diff区域/区域中有两个集群，两者都有上面列出的部署错误。2口香糖不好。

Name:         ml-pipeline-65479485c8-mcj9x
Namespace:    default
Priority:     0
Node:         gke-cluster-3-default-pool-007784cb-qcsn/10.150.0.2
Start Time:   Thu, 17 Sep 2020 22:15:19 +0000
Labels:       app=ml-pipeline
              app.kubernetes.io/name=kubeflow-pipelines-3
              pod-template-hash=65479485c8
Annotations:  kubernetes.io/limit-ranger: LimitRanger plugin set: cpu request for container ml-pipeline-api-server

Status:       Running
IP:           10.4.0.8
IPs:
IP:           10.4.0.8
Controlled By:  ReplicaSet/ml-pipeline-65479485c8
Containers:
  ml-pipeline-api-server:
    Container ID:   ...
    Image:          ...
    Image ID:       ...
    Ports:          8888/TCP, 8887/TCP
    Host Ports:     0/TCP, 0/TCP
    State:          Running
      Started:      Fri, 18 Sep 2020 10:27:31 +0000
    Last State:     Terminated
      Reason:       Error
      Exit Code:    255
      Started:      Fri, 18 Sep 2020 10:20:38 +0000
      Finished:     Fri, 18 Sep 2020 10:27:31 +0000
    Ready:          False
    Restart Count:  98
    Requests:
      cpu:      100m
    Liveness:   exec [wget -q -S -O - http://localhost:8888/apis/v1beta1/healthz] delay=3s timeout=2s period=5s #success=1 #failure=3
    Readiness:  exec [wget -q -S -O - http://localhost:8888/apis/v1beta1/healthz] delay=3s timeout=2s period=5s #success=1 #failure=3
    Environment:
      HAS_DEFAULT_BUCKET:                   true
      BUCKET_NAME:
      PROJECT_ID:                           <set to the key 'project_id' of config map 'gcp-default-config'>  Optional: false
      POD_NAMESPACE:                        default (v1:metadata.namespace)
      DEFAULTPIPELINERUNNERSERVICEACCOUNT:  pipeline-runner
      OBJECTSTORECONFIG_SECURE:             false
      OBJECTSTORECONFIG_BUCKETNAME:
      DBCONFIG_DBNAME:                      kubeflow_pipelines_3_pipeline
      DBCONFIG_USER:                        <set to the key 'username' in secret 'mysql-credential'>  Optional: false
      DBCONFIG_PASSWORD:                    <set to the key 'password' in secret 'mysql-credential'>  Optional: false
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from ml-pipeline-token-77xl8 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  ml-pipeline-token-77xl8:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  ml-pipeline-token-77xl8
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                   From                                               Message
  ----     ------     ----                  ----                                               -------
  Warning  BackOff    52m (x409 over 11h)   kubelet, gke-cluster-3-default-pool-007784cb-qcsn  Back-off restarting failed container
  Warning  Unhealthy  31m (x94 over 12h)    kubelet, gke-cluster-3-default-pool-007784cb-qcsn  Readiness probe failed:
  Warning  Unhealthy  31m (x29 over 10h)    kubelet, gke-cluster-3-default-pool-007784cb-qcsn  (combined from similar events): Readiness probe failed: c
annot exec in a stopped state: unknown
  Warning  Unhealthy  17m (x95 over 12h)    kubelet, gke-cluster-3-default-pool-007784cb-qcsn  Liveness probe failed:
  Normal   Pulled     7m26s (x97 over 12h)  kubelet, gke-cluster-3-default-pool-007784cb-qcsn  Container image "gcr.io/cloud-marketplace/google-cloud-ai
-platform/kubeflow-pipelines/apiserver:1.0.0" already present on machine
  Warning  Unhealthy  75s (x78 over 12h)    kubelet, gke-cluster-3-default-pool-007784cb-qcsn  Liveness probe errored: rpc error: code = DeadlineExceede
d desc = context deadline exceeded

另一个吊舱：

Name:         ml-pipeline-persistenceagent-67db8b8964-mlbmv
Events:
  Type     Reason   Age                   From                                               Message
  ----     ------   ----                  ----                                               -------
  Warning  BackOff  32s (x2238 over 12h)  kubelet, gke-cluster-3-default-pool-007784cb-qcsn  Back-off restarting failed container

溶液

不要让谷歌处理任何存储。取消选中“使用托管存储”并手动设置您自己的工件集合。您实际上不需要在这些字段中输入任何内容，因为管道无论如何都会启动。

Answer 1

Does not have minimum availability错误是泛型的。可能会有很多问题触发它。你需要更深入的分析，才能找到实际的问题。以下是一些可能的原因：

• 资源不足：检查节点是否有足够的资源(CPU/内存)。如果Node比检查Pod的状态好。
• 活动探测和/或就绪探测失败：执行kubectl describe pod <pod-name>检查它们是否失败以及原因。
• 部署错误配置：检查您的部署yaml文件，以查看是否有任何错误或遗留下来的以前配置。
• 您还可以尝试稍候，因为有时需要一些时间来部署所有内容和/或尝试更改区域/区域。