无法完全删除Kubernetes CustomResource

Question

我在删除自定义资源定义时遇到了问题。我正在尝试将kubeless从v1.0.0-alpha.7 7升级到v1.0.0-alpha.8.

我尝试删除所有创建的自定义资源。

$ kubectl delete -f kubeless-v1.0.0-alpha.7.yaml
deployment "kubeless-controller-manager" deleted
serviceaccount "controller-acct" deleted
clusterrole "kubeless-controller-deployer" deleted
clusterrolebinding "kubeless-controller-deployer" deleted
customresourcedefinition "functions.kubeless.io" deleted
customresourcedefinition "httptriggers.kubeless.io" deleted
customresourcedefinition "cronjobtriggers.kubeless.io" deleted
configmap "kubeless-config" deleted

但当我试着

$ kubectl get customresourcedefinition
NAME                    AGE
functions.kubeless.io   21d

正因为如此，当我下一次尝试通过做来升级时，我明白了，

$ kubectl create -f kubeless-v1.0.0-alpha.8.yaml
Error from server (AlreadyExists): error when creating "kubeless-v1.0.0-alpha.8.yaml": object is being deleted: customresourcedefinitions.apiextensions.k8s.io "functions.kubeless.io" already exists

我认为由于函数定义中的这种不匹配，hello world示例失败了。

$ kubeless function deploy hellopy --runtime python2.7 --from-file test.py --handler test.hello
INFO[0000] Deploying function...
FATA[0000] Failed to deploy hellopy. Received:
the server does not allow this method on the requested resource (post functions.kubeless.io)

最后，这里是

$ kubectl describe customresourcedefinitions.apiextensions.k8s.io
Name:         functions.kubeless.io
Namespace:
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"apiextensions.k8s.io/v1beta1","description":"Kubernetes Native Serverless Framework","kind":"CustomResourceDefinition","metadata":{"anno...
API Version:  apiextensions.k8s.io/v1beta1
Kind:         CustomResourceDefinition
Metadata:
  Creation Timestamp:             2018-08-02T17:22:07Z
  Deletion Grace Period Seconds:  0
  Deletion Timestamp:             2018-08-24T17:15:39Z
  Finalizers:
    customresourcecleanup.apiextensions.k8s.io
  Generation:        1
  Resource Version:  99792247
  Self Link:         /apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/functions.kubeless.io
  UID:               951713a6-9678-11e8-bd68-0a34b6111990
Spec:
  Group:  kubeless.io
  Names:
    Kind:       Function
    List Kind:  FunctionList
    Plural:     functions
    Singular:   function
  Scope:        Namespaced
  Version:      v1beta1
Status:
  Accepted Names:
    Kind:       Function
    List Kind:  FunctionList
    Plural:     functions
    Singular:   function
  Conditions:
    Last Transition Time:  2018-08-02T17:22:07Z
    Message:               no conflicts found
    Reason:                NoConflicts
    Status:                True
    Type:                  NamesAccepted
    Last Transition Time:  2018-08-02T17:22:07Z
    Message:               the initial names have been accepted
    Reason:                InitialNamesAccepted
    Status:                True
    Type:                  Established
    Last Transition Time:  2018-08-23T13:29:45Z
    Message:               CustomResource deletion is in progress
    Reason:                InstanceDeletionInProgress
    Status:                True
    Type:                  Terminating
Events:                    <none>

Answer 1

因此，根本原因是带有终结器的自定义资源可以“死锁”。CustomResource "functions.kubeless.io“有一个

Finalizers: customresourcecleanup.apiextensions.k8s.io

而这可能会使它在删除时处于一个糟糕的状态。

https://github.com/kubernetes/kubernetes/issues/60538

我遵循了这个解决办法中提到的步骤，现在它被删除了。希望这能帮到其他遇到这个问题的人。

Answer 2

$ kubectl get crd

NAME                                                            CREATED AT
accesscontrolpolicies.networking.zephyr.solo.io                 2020-04-22T12:58:39Z
istiooperators.install.istio.io                                 2020-04-22T13:49:20Z
kubernetesclusters.discovery.zephyr.solo.io                     2020-04-22T12:58:39Z
meshes.discovery.zephyr.solo.io                                 2020-04-22T12:58:39Z
meshservices.discovery.zephyr.solo.io                           2020-04-22T12:58:39Z
meshworkloads.discovery.zephyr.solo.io                          2020-04-22T12:58:39Z
trafficpolicies.networking.zephyr.solo.io                       2020-04-22T12:58:39Z
virtualmeshcertificatesigningrequests.security.zephyr.solo.io   2020-04-22T12:58:39Z
virtualmeshes.networking.zephyr.solo.io                         2020-04-22T12:58:39Z

$ kubectl delete crd istiooperators.install.istio.io

delete error

$ kubectl patch crd/istiooperators.install.istio.io -p '{"metadata":{"finalizers":[]}}' --type=merge
success delete crd istiooperators.install.istio.io

结果

NAME                                                            CREATED AT
accesscontrolpolicies.networking.zephyr.solo.io                 2020-04-22T12:58:39Z
kubernetesclusters.discovery.zephyr.solo.io                     2020-04-22T12:58:39Z
meshes.discovery.zephyr.solo.io                                 2020-04-22T12:58:39Z
meshservices.discovery.zephyr.solo.io                           2020-04-22T12:58:39Z
meshworkloads.discovery.zephyr.solo.io                          2020-04-22T12:58:39Z
trafficpolicies.networking.zephyr.solo.io                       2020-04-22T12:58:39Z
virtualmeshcertificatesigningrequests.security.zephyr.solo.io   2020-04-22T12:58:39Z
virtualmeshes.networking.zephyr.solo.io                         2020-04-22T12:58:39Z

Answer 3

尝试：

oc patch some.crd/crd_name -p '{"metadata":{"finalizers":[]}}' --type=merge

解决了我的问题后，试图强迫删除卡住了。