使用DaemonSet运行glusterfs集群

Question

我一直试图在我的kubernetes集群上运行一个glusterfs集群，使用这些：

glusterfs-service.json

{
  "kind": "Service",
  "apiVersion": "v1",
  "metadata": {
    "name": "glusterfs-cluster"
  },
  "spec": {
    "type": "NodePort",
    "selector": {
      "name": "gluster"
    },
    "ports": [
      {
        "port": 1
      }
    ]
  }
}

和glusterfs-server.json

{
  "apiVersion": "extensions/v1beta1",
  "kind": "DaemonSet",
  "metadata": {
    "labels": {
      "name": "gluster"
    },
    "name": "gluster"
  },
  "spec": {
    "selector": {
      "matchLabels": {
        "name": "gluster"
      }
    },
    "template": {
      "metadata": {
        "labels": {
          "name": "gluster"
        }
      },
      "spec": {
        "containers": [
          {
            "name": "gluster",
            "image": "gluster/gluster-centos",
            "livenessProbe": {
              "exec": {
                "command": [
                  "/bin/bash",
                  "-c",
                  "systemctl status glusterd.service"
                ]
              }
            },
            "readinessProbe": {
              "exec": {
                "command": [
                  "/bin/bash",
                  "-c",
                  "systemctl status glusterd.service"
                ]
              }
            },
            "securityContext": {
              "privileged": true
            },
            "volumeMounts": [
              {
                "mountPath": "/mnt/brick1",
                "name": "gluster-brick"
              },
              {
                "mountPath": "/etc/gluster",
                "name": "gluster-etc"
              },
              {
                "mountPath": "/var/log/gluster",
                "name": "gluster-logs"
              },
              {
                "mountPath": "/var/lib/glusterd",
                "name": "gluster-config"
              },
              {
                "mountPath": "/dev",
                "name": "gluster-dev"
              },
              {
                "mountPath": "/sys/fs/cgroup",
                "name": "gluster-cgroup"
              }
            ]
          }
        ],
        "dnsPolicy": "ClusterFirst",
        "hostNetwork": true,
        "volumes": [
          {
            "hostPath": {
              "path": "/mnt/brick1"
            },
            "name": "gluster-brick"
          },
          {
            "hostPath": {
              "path": "/etc/gluster"
            },
            "name": "gluster-etc"
          },
          {
            "hostPath": {
              "path": "/var/log/gluster"
            },
            "name": "gluster-logs"
          },
          {
            "hostPath": {
              "path": "/var/lib/glusterd"
            },
            "name": "gluster-config"
          },
          {
            "hostPath": {
              "path": "/dev"
            },
            "name": "gluster-dev"
          },
          {
            "hostPath": {
              "path": "/sys/fs/cgroup"
            },
            "name": "gluster-cgroup"
          }
        ]
      }
    }
  }
}

根据我的吊舱定义，我要做的是：

"volumes": [
  {
    "name": "< volume name >",
    "glusterfs": {
      "endpoints": "glusterfs-cluster.default.svc.cluster.local",
      "path": "< gluster path >",
      "readOnly": false
    }
  }
]

但是豆荚的制作是定时的，因为它不能预先处理音量

看起来也只有一个臀部豆荚在运行

这是我的日志：http://imgur.com/a/j2I8r

然后，我尝试在与我的gluster集群运行相同的名称空间上运行我的荚，现在我得到了这个错误：

Operation for "\"kubernetes.io/glusterfs/01a0834e-64ab-11e6-af52-42010a840072-ssl-certificates\" (\"01a0834e-64ab-11e6-af52-42010a840072\")" failed.
No retries permitted until 2016-08-17 18:51:20.61133778 +0000 UTC (durationBeforeRetry 2m0s).
Error: MountVolume.SetUp failed for volume "kubernetes.io/glusterfs/01a0834e-64ab-11e6-af52-42010a840072-ssl-certificates" (spec.Name: "ssl-certificates") pod "01a0834e-64ab-11e6-af52-42010a840072" (UID: "01a0834e-64ab-11e6-af52-42010a840072") with: glusterfs: mount failed:
mount failed: exit status 1
Mounting arguments:
10.132.0.7:ssl_certificates /var/lib/kubelet/pods/01a0834e-64ab-11e6-af52-42010a840072/volumes/kubernetes.io~glusterfs/ssl-certificates
glusterfs [log-level=ERROR log-file=/var/lib/kubelet/plugins/kubernetes.io/glusterfs/ssl-certificates/caddy-server-1648321103-epvdi-glusterfs.log]
Output: Mount failed. Please check the log file for more details. the following error information was pulled from the glusterfs log to help diagnose this issue:
[2016-08-17 18:49:20.583585] E [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-mgmt: failed to fetch volume file (key:ssl_certificates)
[2016-08-17 18:49:20.610531] E [glusterfsd-mgmt.c:1494:mgmt_getspec_cbk] 0-glusterfs: failed to get the 'volume file' from server

Answer 1

日志清楚地说明了发生了什么：

未能获得端点

因为：

  "ports": [
  {
    "port": 1
  }

在几个方面都是假的。首先，一个"1“的港口很可疑。其次，它在DaemonSet端没有匹配的DaemonSet，kubernetes可以指向这个Service --因此，它不会为(podIP, protocol, port)元组创建Endpoints。因为glusterfs (合理地)希望直接与底层的Pod联系，而不通过Service，所以它无法发现Pod，一切都突然停止。