CrashLoopBackOff (postgres) - GCP

Question

​

​

来自服务器的

错误:获取https://10.128.15.203:10250/containerLogs/default/postgres-54db6bdb8b-cmrsb/postgres：EOF

我怎样才能解决这个问题？什么才是理由。我使用这个教程来配置所有的东西(https://severalnines.com/database-blog/using-kubernetes-deploy-postgresql)。如果你需要更多的信息，请告诉我！

kubectl描述pods postgres-54db6bdb8b-cmrsb

Name:           postgres-54db6bdb8b-cmrsb
Namespace:      default
Priority:       0
Node:           gke-booknotes-pool-2-c1d23e62-r6nb/10.128.15.203
Start Time:     Sat, 14 Dec 2019 23:27:20 +0700
Labels:         app=postgres
                pod-template-hash=54db6bdb8b
Annotations:    kubernetes.io/limit-ranger: LimitRanger plugin set: cpu request for container postgres
Status:         Running
IP:             10.56.1.3
IPs:            <none>
Controlled By:  ReplicaSet/postgres-54db6bdb8b
Containers:
  postgres:
    Container ID:   docker://1a607cfb9a8968d708ff79419ec8bfc7233fb5ad29fb1055034ddaacfb793d6a
    Image:          postgres:10.4
    Image ID:       docker-pullable://postgres@sha256:9625c2fb34986a49cbf2f5aa225d8eb07346f89f7312f7c0ea19d82c3829fdaa
    Port:           5432/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       ContainerCannotRun
      Message:      error while creating mount source path '/mnt/data': mkdir /mnt/data: read-only file system
      Exit Code:    128
      Started:      Sat, 14 Dec 2019 23:54:00 +0700
      Finished:     Sat, 14 Dec 2019 23:54:00 +0700
    Ready:          False
    Restart Count:  25
    Requests:
      cpu:  100m
    Environment Variables from:
      postgres-config  ConfigMap  Optional: false
    Environment:       <none>
    Mounts:
      /var/lib/postgresql/data from postgredb (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-t48dw (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  postgredb:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  postgres-pv-claim
    ReadOnly:   false
  default-token-t48dw:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-t48dw
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                     From                                         Message
  ----     ------     ----                    ----                                         -------
  Normal   Scheduled  32m                     default-scheduler                            Successfully assigned default/postgres-54db6bdb8b-cmrsb to gke-booknotes-pool-2-c1d23e62-r6nb
  Normal   Pulled     28m (x5 over 30m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Container image "postgres:10.4" already present on machine
  Normal   Created    28m (x5 over 30m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Created container postgres
  Warning  Failed     28m (x5 over 30m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Error: failed to start container "postgres": Error response from daemon: error while creating mount source path '/mnt/data': mkdir /mnt/data: read-only file system
  Warning  BackOff    27m (x10 over 29m)      kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Back-off restarting failed container
  Warning  Failed     23m (x4 over 25m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Error: failed to start container "postgres": Error response from daemon: error while creating mount source path '/mnt/data': mkdir /mnt/data: read-only file system
  Warning  BackOff    22m (x11 over 25m)      kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Back-off restarting failed container
  Normal   Pulled     22m (x5 over 25m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Container image "postgres:10.4" already present on machine
  Normal   Created    22m (x5 over 25m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Created container postgres
  Normal   Pulled     19m (x4 over 20m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Container image "postgres:10.4" already present on machine
  Normal   Created    19m (x4 over 20m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Created container postgres
  Warning  Failed     19m (x4 over 20m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Error: failed to start container "postgres": Error response from daemon: error while creating mount source path '/mnt/data': mkdir /mnt/data: read-only file system
  Warning  BackOff    18m (x11 over 20m)      kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Back-off restarting failed container
  Normal   Created    15m (x4 over 17m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Created container postgres
  Warning  Failed     15m (x4 over 17m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Error: failed to start container "postgres": Error response from daemon: error while creating mount source path '/mnt/data': mkdir /mnt/data: read-only file system
  Normal   Pulled     14m (x5 over 17m)       kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Container image "postgres:10.4" already present on machine
  Warning  BackOff    12m (x19 over 17m)      kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Back-off restarting failed container
  Normal   Pulled     5m38s (x5 over 8m29s)   kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Container image "postgres:10.4" already present on machine
  Normal   Created    5m38s (x5 over 8m27s)   kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Created container postgres
  Warning  Failed     5m37s (x5 over 8m24s)   kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Error: failed to start container "postgres": Error response from daemon: error while creating mount source path '/mnt/data': mkdir /mnt/data: read-only file system
  Warning  BackOff    5m24s (x10 over 7m58s)  kubelet, gke-booknotes-pool-2-c1d23e62-r6nb  Back-off restarting failed container

这里还有我的yaml文件：

deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:10.4
          imagePullPolicy: "IfNotPresent"
          ports:
            - containerPort: 5432
          envFrom:
            - configMapRef:
                name: postgres-config
          volumeMounts:
            - mountPath: /var/lib/postgresql/data
              name: postgredb
      volumes:
        - name: postgredb
          persistentVolumeClaim:
            claimName: postgres-pv-claim

postgres-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: postgres-config
  labels:
    app: postgres
data:
  POSTGRES_DB: postgresdb
  POSTGRES_USER: postgresadmin
  POSTGRES_PASSWORD: some_password

postgres-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: postgres
  labels:
    app: postgres
spec:
  type: NodePort
  ports:
   - port: 5432
  selector:
   app: postgres

postgres-storage.yaml

kind: PersistentVolume
apiVersion: v1
metadata:
  name: postgres-pv-volume
  labels:
    type: local
    app: postgres
spec:
  storageClassName: manual
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: postgres-pv-claim
  labels:
    app: postgres
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

在我将RWM更改为RWO之后-然后我就这样做了(我删除了旧的实例并创建了新的实例)：

Name:           postgres-54db6bdb8b-wgvr2
Namespace:      default
Priority:       0
Node:           gke-booknotes-pool-1-3e566443-dc08/10.128.15.236
Start Time:     Sun, 15 Dec 2019 04:56:57 +0700
Labels:         app=postgres
                pod-template-hash=54db6bdb8b
Annotations:    kubernetes.io/limit-ranger: LimitRanger plugin set: cpu request for container postgres
Status:         Running
IP:             10.56.6.13
IPs:            <none>
Controlled By:  ReplicaSet/postgres-54db6bdb8b
Containers:
  postgres:
    Container ID:   docker://1070018c2a670cc7e0248e6269c271c3cba022fdd2c9cc5099a8eb4da44f7d65
    Image:          postgres:10.4
    Image ID:       docker-pullable://postgres@sha256:9625c2fb34986a49cbf2f5aa225d8eb07346f89f7312f7c0ea19d82c3829fdaa
    Port:           5432/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       ContainerCannotRun
      Message:      error while creating mount source path '/mnt/data': mkdir /mnt/data: read-only file system
      Exit Code:    128
      Started:      Sun, 15 Dec 2019 10:56:21 +0700
      Finished:     Sun, 15 Dec 2019 10:56:21 +0700
    Ready:          False
    Restart Count:  76
    Requests:
      cpu:  100m
    Environment Variables from:
      postgres-config  ConfigMap  Optional: false
    Environment:       <none>
    Mounts:
      /var/lib/postgresql/data from postgredb (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-t48dw (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  postgredb:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  postgres-pv-claim
    ReadOnly:   false
  default-token-t48dw:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-t48dw
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason   Age                  From                                         Message
  ----     ------   ----                 ----                                         -------
  Warning  BackOff  81s (x1629 over 6h)  kubelet, gke-booknotes-pool-1-3e566443-dc08  Back-off restarting failed container

Answer 1

1.如何修复CrashLoopBackOff (postgres) - GCP

问题是:创建挂载源路径‘/mnt/data’时出错: mkdir /mnt/data：只读文件系统。

您需要确保postgres-pv-claim是可写的。您需要使用RWO访问重新创建pv和pv声明(您肯定是将其错误地输入为RO，这就是您遇到问题的原因)，然后尝试部署postgres，这将解决问题。

2.修复FailedScheduling 69s (x10超过7m35s)默认-调度程序pod有未绑定的即时PersistentVolumeClaims (重复4次)

在GKE中提供持久卷--您不需要创建PersistentVolume对象--它们是由GKE动态创建的。所以解决Warning FailedScheduling 69s (x10 over 7m35s)问题

1. 从pvc中删除storageClassName属性，
2. 删除pv

这应该能解决问题。请参阅下面修订的postgres-storage.yaml。

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: postgres-pv-claim
  labels:
    app: postgres
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi