当我重新启动一个吊舱时，它会显示“音量已由pod /minio-3附加”。

Question

当我用“kubectl删除-n minio pod 3”重新启动一个荚时，kubelet显示“音量已由pod /minio-3附加”。似乎这卷书是附在一个旧的上。我怎样才能让它发挥作用。

[root@control01 ~]# kubectl get pod -n minio 
NAME      READY   STATUS              RESTARTS   AGE
minio-0   0/1     ContainerCreating   0          62m
minio-1   1/1     Running             0          128m
minio-2   1/1     Running             1          6d7h
minio-3   0/1     ContainerCreating   0          96m

[root@control12 ~]# cat /var/log/messages |grep 'Sep 13'
Sep 13 16:48:10 control12 kubelet: E0913 16:48:10.189343   40141 nestedpendingoperations.go:270] Operation for "\"flexvolume-ceph.rook.io/rook-ceph/e51e0fbd-f09f-430a-8b47-2ca1dbdfdc2e-pvc-06a22ce4-cbbb-4cd7-82c5-d7bf9755fbd9\" (\"e51e0fbd-f09f-430a-8b47-2ca1dbdfdc2e\")" failed. No retries permitted until 2022-09-13 16:50:12.189281072 +0800 CST m=+5554.416909175 (durationBeforeRetry 2m2s). Error: "MountVolume.SetUp failed for volume \"pvc-06a22ce4-cbbb-4cd7-82c5-d7bf9755fbd9\" (UniqueName: \"flexvolume-ceph.rook.io/rook-ceph/e51e0fbd-f09f-430a-8b47-2ca1dbdfdc2e-pvc-06a22ce4-cbbb-4cd7-82c5-d7bf9755fbd9\") pod \"minio-3\" (UID: \"e51e0fbd-f09f-430a-8b47-2ca1dbdfdc2e\") : mount command failed, status: Failure, reason: Rook: Mount volume failed: failed to attach volume pvc-06a22ce4-cbbb-4cd7-82c5-d7bf9755fbd9 for pod minio/minio-3. Volume is already attached by pod minio/minio-3. Status Pending"

[root@control01 ~]# kubectl describe pod -n minio minio-3
Name:           minio-3
Namespace:      minio
Priority:       0
Node:           control12/192.168.1.112
Start Time:     Tue, 13 Sep 2022 15:18:28 +0800
Labels:         app=minio
                controller-revision-hash=minio-95c8c444c
                statefulset.kubernetes.io/pod-name=minio-3
Annotations:    <none>
Status:         Pending
IP:             
IPs:            <none>
Controlled By:  StatefulSet/minio
Containers:
  minio:
    Container ID:  
    Image:         minio/minio:RELEASE.2021-04-06T23-11-00Z
    Image ID:      
    Port:          9000/TCP
    Host Port:     0/TCP
    Args:
      server
      http://minio-{0...3}.minio.minio.svc.cluster.local/minio/data
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
   ...
   ...
Events:
  Type     Reason       Age                  From                Message
  ----     ------       ----                 ----                -------
  Warning  FailedMount  52m (x13 over 93m)   kubelet, control12  Unable to attach or mount volumes: unmounted volumes=[minio-data], unattached volumes=[minio-data tz-config default-token-np5x5]: timed out waiting for the condition
  Warning  FailedMount  11m (x50 over 97m)   kubelet, control12  MountVolume.SetUp failed for volume "pvc-06a22ce4-cbbb-4cd7-82c5-d7bf9755fbd9" : mount command failed, status: Failure, reason: Rook: Mount volume failed: failed to attach volume pvc-06a22ce4-cbbb-4cd7-82c5-d7bf9755fbd9 for pod minio/minio-3. Volume is already attached by pod minio/minio-3. Status Pending
  Warning  FailedMount  6m36s (x7 over 95m)  kubelet, control12  Unable to attach or mount volumes: unmounted volumes=[minio-data], unattached volumes=[tz-config default-token-np5x5 minio-data]: timed out waiting for the condition
  Warning  FailedMount  2m1s (x9 over 86m)   kubelet, control12  Unable to attach or mount volumes: unmounted volumes=[minio-data], unattached volumes=[default-token-np5x5 minio-data tz-config]: timed out waiting for the condition

Answer 1

如果要删除附件：

$ kubectl get pv
# find the pvc NAME by CLAIM 
$ kubectl get volumeattachment
# use the NAME from above as PV in the output to look up CSI NAME
$ kubectl delete volumeattachment [CSI NAME]