为什么EMQX持久性在本地kubernetes的工作中不适用于蔚蓝的kubernetes？

Question

在本地机器上使用kubernetes(minikube)状态集时，EMQX规则是持久化的，因为相同的pod被分配给emqx节点，例如/opt/emqx/data/mnesia/emqx@172.17.0.9.。即使在新的吊舱启动时，我删除了吊舱，它也会得到与以前相同的IP。一切都正常运转。

但是，当我使用 aks (azure kubernetes)使用azure文件在aks集群上部署EMQX时，pod每次都是不同的。例如，如果/opt/emqx/data/mnesia/emqx@10.1.1.10被分配给EMQX节点，那么如果我尝试删除pod，那么/opt/emqx/data/mnesia/emqx@10.1.1.11可能被分配给它。

所以，没有什么是持久的。

局部代码

---

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-storage5
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

---

apiVersion: v1
kind: PersistentVolume
metadata:
  name: emqx-pv5
spec:
  capacity:
    storage: 300Mi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage5
  local:
    path: /opt/emqx/data/mnesia
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - minikube

---

apiVersion: v1
kind: Service
metadata:
  name: emqx-headless
spec:
  type: ClusterIP
  clusterIP: None
  selector:
    app: emqx
  ports:
    - name: mqtt
      port: 1883
      protocol: TCP
      targetPort: 1883
    - name: mqttssl
      port: 8883
      protocol: TCP
      targetPort: 8883
    - name: mgmt
      port: 8081
      protocol: TCP
      targetPort: 8081
    - name: websocket
      port: 8083
      protocol: TCP
      targetPort: 8083
    - name: wss
      port: 8084
      protocol: TCP
      targetPort: 8084
    - name: dashboard
      port: 18083
      protocol: TCP
      targetPort: 18083

---

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: emqx-statefulset
  labels:
    app: emqx
spec:
  replicas: 1
  serviceName: emqx-headless
  selector:
    matchLabels:
      app: emqx
  template:
    metadata:
      labels:
        app: emqx
    spec:
      containers:
        - name: emqx
          image: emqx/emqx:4.2.7
          ports:
            - name: emqx-dashboard
              containerPort: 18083
            - name: ssl-port
              containerPort: 8883
            - name: emqx-port
              containerPort: 1883
            - name: ssl-dashboard
              containerPort: 18084
          env:
            - name: EMQX_LOADED_PLUGINS
              value: emqx_management,emqx_recon,emqx_retainer,emqx_dashboard,emqx_rule_engine,emqx_auth_username
            - name: EMQX_CLUSTER__DISCOVERY
              value: k8s
            - name: EMQX_NAME
              value: emqx
            - name: EMQX_CLUSTER__K8S__APISERVER
              value: https://kubernetes.default:443
            - name: EMQX_CLUSTER__K8S__SERVICE_NAME
              value: emqx
            - name: EMQX_CLUSTER__K8S__ADDRESS_TYPE
              value: ip
            - name: EMQX_CLUSTER__K8S__APP_NAME
              value: emqx
            - name: EMQX_ALLOW_ANONYMOUS
              value: "false"
            - name: EMQX_LISTENER__SSL__EXTERNAL__MAX_CONNECTIONS
              value: "1024000"
            - name: EMQX_AUTH__USER__PASSWORD_HASH
              value: sha256
            - name: EMQX_AUTH__USER__1__USERNAME
              value: 
            - name: EMQX_AUTH__USER__1__PASSWORD
              value: 
            - name: EMQX_DASHBOARD__DEFAULT_USER__LOGIN
              value: 
            - name: EMQX_DASHBOARD__DEFAULT_USER__PASSWORD
              value:
            - name: EMQX_DASHBOARD__LISTENER__HTTPS
              value: "18084"
            - name: MQX_DASHBOARD__LISTENER__HTTPS__ACCEPTORS
              value: "4"
            - name: EMQX_DASHBOARD__LISTENER__HTTPS__MAX_CLIENTS
              value: "512"
          tty: true
          volumeMounts:
            - name: emqx-mnesia
              mountPath: "/opt/emqx/data/mnesia"

  volumeClaimTemplates:
    - metadata:
        name: emqx-mnesia
      spec:
        accessModes: [ "ReadWriteOnce" ]
        storageClassName: "local-storage5"
        resources:
          requests:
            storage: 300Mi

Azure Kubernetes码

apiVersion: v1
kind: ServiceAccount
metadata:
  name: emqx
  namespace: emqx-test

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: emqx
subjects:
  - kind: ServiceAccount
    name: emqx
    namespace: emqx-test
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: emqx-files
provisioner: kubernetes.io/azure-file
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=0
  - gid=0
  - mfsymlinks
  - cache=strict
  - actimeo=30
parameters:
  skuName: Standard_LRS
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: emqx-pvc
  namespace: emqx-test
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: emqx-files
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
  name: emqx
  namespace: emqx-test
spec:
  ports:
    - name: emqx-dashboard
      port: 80
      targetPort: 18083
      protocol: TCP
    - name: ssl-port
      port: 8883
      targetPort: ssl-port
      protocol: TCP
    - name: emqx-port
      port: 1883
      targetPort: emqx-port
      protocol: TCP
    - name: ssl-dashboard
      port: 443
      targetPort: 18084
      protocol: TCP
  selector:
    app: emqx
  type: LoadBalancer
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: emqx
  labels:
    app: emqx
  namespace: emqx-test
spec:
  serviceName: "emqx"
  selector:
    matchLabels:
      app: emqx
  replicas: 1
  template:
    metadata:
      labels:
        app: emqx
    spec:
      containers:
        - name: emqx
          image: emqx/emqx:4.2.7
          ports:
            - name: emqx-dashboard
              containerPort: 18083
            - name: ssl-port
              containerPort: 8883
            - name: emqx-port
              containerPort: 1883
            - name: ssl-dashboard
              containerPort: 18084
          env:
            - name: EMQX_LOADED_PLUGINS
              value: emqx_management,emqx_recon,emqx_retainer,emqx_dashboard,emqx_rule_engine,emqx_auth_username
            - name: EMQX_CLUSTER__DISCOVERY
              value: k8s
            - name: EMQX_NAME
              value: emqx
            - name: EMQX_CLUSTER__K8S__APISERVER
              value: https://kubernetes.default:443
            - name: EMQX_CLUSTER__K8S__NAMESPACE
              value: emqx-test
            - name: EMQX_CLUSTER__K8S__SERVICE_NAME
              value: emqx
            - name: EMQX_CLUSTER__K8S__ADDRESS_TYPE
              value: ip
            - name: EMQX_CLUSTER__K8S__APP_NAME
              value: emqx
            - name: EMQX_ALLOW_ANONYMOUS
              value: "false"
            - name: EMQX_LISTENER__SSL__EXTERNAL__MAX_CONNECTIONS
              value: "1024000"
            - name: EMQX_AUTH__USER__PASSWORD_HASH
              value: sha256
            - name: EMQX_AUTH__USER__1__USERNAME
              value:
            - name: EMQX_AUTH__USER__1__PASSWORD
              value:
            - name: EMQX_DASHBOARD__DEFAULT_USER__LOGIN
              value:
            - name: EMQX_DASHBOARD__DEFAULT_USER__PASSWORD
              value:
            - name: EMQX_DASHBOARD__LISTENER__HTTPS
              value: "18084"
            - name: MQX_DASHBOARD__LISTENER__HTTPS__ACCEPTORS
              value: "4"
            - name: EMQX_DASHBOARD__LISTENER__HTTPS__MAX_CLIENTS
              value: "512"
          volumeMounts:
            - name: emqx-data
              mountPath: "/opt/emqx/data/mnesia"
          tty: true
      volumes:
        - name: emqx-data
          persistentVolumeClaim:
            claimName: emqx-pvc

Answer 1

在关于k8s的StatefulSet基础文档中，您可以看到：

的序号、主机名、SRV记录和记录名都没有改变，但是与Pods相关联的IP地址可能已经更改了。在本教程使用的集群中，它们已经使用了。--这就是为什么不配置其他应用程序来通过IP地址连接到StatefulSet中的Pods的原因。

这是预期的，正如您所看到的，文档中提到了这种行为。

但是，为什么你看到不同的行为在迷你和不同的天青？IP地址由CNI分配。在迷你默认的CNI，它的码头-桥，并在天天其Azure CNI，所以它是由CNI指定的地址。

最好总是假设不能依靠pod地址来保持静态。使用DNS、用于状态集和其他吊舱和服务进行通信，从不直接使用硬编码的pod ip地址。