如何从另一个集群pod连接kubernetes集群上的mongo副本集

Question

我有一个在kubernetes集群(在AWS EKS上)上运行的mongo db副本集，比如cluster-1。它在cidr为192.174.0.0/16的VPC-1内运行。

我在一个单独的VPC中有另一个集群，比如VPC-2，我将在mongo集群上运行一些应用程序。此私有网络cidr范围为192.176.0.0/16，所有私有网络对等和安全组出入口规则运行正常，我能够ping通两个私有网络中的集群节点。

我为mongo集群使用了NodePort类型的服务和StatefulSet：

apiVersion: v1
kind: Service
metadata:
  name: mongodb-service
  labels:
    name: mongodb
spec:
  selector:
    role: mongo
  type: NodePort
  ports:
  - port: 26017
    targetPort: 27017
    nodePort: 30017

以下是mongo集群cluster-1中的节点和pod：

ubuntu@ip-192-174-5-253:/st_config/kubeobj$ kubectl get nodes -o wide
NAME                                             STATUS   ROLES    AGE   VERSION              INTERNAL-IP       EXTERNAL-IP      OS-IMAGE         KERNEL-VERSION                  CONTAINER-RUNTIME
ip-192-174-187-133.ap-south-1.compute.internal   Ready    <none>   19h   v1.16.8-eks-e16311   192.174.187.133   13.232.195.39    Amazon Linux 2   4.14.181-140.257.amzn2.x86_64   docker://19.3.6
ip-192-174-23-229.ap-south-1.compute.internal    Ready    <none>   19h   v1.16.8-eks-e16311   192.174.23.229    13.234.111.139   Amazon Linux 2   4.14.181-140.257.amzn2.x86_64   docker://19.3.6
ubuntu@ip-192-174-5-253:/st_config/kubeobj$


ubuntu@ip-192-174-5-253:/st_config/kubeobj$ kubectl get pods -o wide
NAME       READY   STATUS    RESTARTS   AGE   IP                NODE                                             NOMINATED NODE   READINESS GATES
mongod-0   1/1     Running   0          45m   192.174.8.10      ip-192-174-23-229.ap-south-1.compute.internal    <none>           <none>
mongod-1   1/1     Running   0          44m   192.174.133.136   ip-192-174-187-133.ap-south-1.compute.internal   <none>           <none>
ubuntu@ip-192-174-5-253:/st_config/kubeobj$

如果我尝试使用一个特定的节点地址进行连接，或者同时使用两个节点地址，kubernetes可能正在以循环方式进行负载平衡或轮换连接：

ubuntu@ip-192-176-42-206:~$ mongo mongodb://192.174.23.229:30017
MongoDB shell version v3.6.3
connecting to: mongodb://192.174.23.229:30017
MongoDB server version: 3.4.24
WARNING: shell and server versions do not match
test_rs0:PRIMARY>

ubuntu@ip-192-176-42-206:~$ mongo mongodb://192.174.23.229:30017
MongoDB shell version v3.6.3
connecting to: mongodb://192.174.23.229:30017
MongoDB server version: 3.4.24
WARNING: shell and server versions do not match
test_rs0:SECONDARY>

ubuntu@ip-192-176-42-206:~$ mongo mongodb://192.174.23.229:30017,192.174.187.133:30017
MongoDB shell version v3.6.3
connecting to: mongodb://192.174.23.229:30017,192.174.187.133:30017
MongoDB server version: 3.4.24
WARNING: shell and server versions do not match
test_rs0:PRIMARY>

我希望利用副本集的功能。因此，当我使用连接字符串作为- mongodb://192.174.23.229:30017,192.174.187.133:30017/?replicaSet=test_rs0时，它实际上是在获取pod的FQDN，这些pod没有从VPC-2中的cluster-2节点/pod中的节点解析。

ubuntu@ip-192-176-42-206:~$ mongo mongodb://192.174.23.229:30017,192.174.187.133:30017/?replicaSet=test_rs0
MongoDB shell version v3.6.3
connecting to: mongodb://192.174.23.229:30017,192.174.187.133:30017/?replicaSet=test_rs0
2020-06-23T15:59:07.407+0000 I NETWORK  [thread1] Starting new replica set monitor for test_rs0/192.174.23.229:30017,192.174.187.133:30017
2020-06-23T15:59:07.409+0000 I NETWORK  [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to 192.174.23.229:30017 (1 connections now open to 192.174.23.229:30017 with a 5 second timeout)
2020-06-23T15:59:07.409+0000 I NETWORK  [thread1] Successfully connected to 192.174.187.133:30017 (1 connections now open to 192.174.187.133:30017 with a 5 second timeout)
2020-06-23T15:59:07.410+0000 I NETWORK  [thread1] changing hosts to test_rs0/mongod-0.mongodb-service.default.svc.cluster.local:27017,mongod-1.mongodb-service.default.svc.cluster.local:27017 from test_rs0/192.174.187.133:30017,192.174.23.229:30017
2020-06-23T15:59:07.415+0000 I NETWORK  [thread1] getaddrinfo("mongod-1.mongodb-service.default.svc.cluster.local") failed: Name or service not known
2020-06-23T15:59:07.415+0000 I NETWORK  [ReplicaSetMonitor-TaskExecutor-0] getaddrinfo("mongod-0.mongodb-service.default.svc.cluster.local") failed: Name or service not known
2020-06-23T15:59:07.917+0000 I NETWORK  [thread1] getaddrinfo("mongod-0.mongodb-service.default.svc.cluster.local") failed: Name or service not known
2020-06-23T15:59:07.918+0000 I NETWORK  [thread1] getaddrinfo("mongod-1.mongodb-service.default.svc.cluster.local") failed: Name or service not known
2020-06-23T15:59:07.918+0000 W NETWORK  [thread1] Unable to reach primary for set test_rs0
2020-06-23T15:59:07.918+0000 I NETWORK  [thread1] Cannot reach any nodes for set test_rs0. Please check network connectivity and the status of the set. This has happened for 1 checks in a row.

我是否需要一些额外的DNS服务才能在VPC-2节点中解析这些名称？最好的方法是什么？

另外，我如何使用连接字符串可以基于服务名称，例如。从VPC-2中的任何节点执行mongodb://mongodb-service.default.svc.cluster.local:/?replicaSet=test_rs0？它可以在VPC-1中的任何pod上运行。但我需要在VPC-2的集群中的pod上运行，这样我就不需要在连接字符串中指定特定的pod/节点IP。我所有的kubernetes对象都在默认名称空间中。

真的很感谢你的帮助。**请注意:我没有使用helm **

Answer 1

Kubernetes有核心连接到每个pod。如果我没记错的话，您可以使用StatefulSet部署。

连接每个mongo集群的最佳方法是使用ClusterIP相互通信。

如果您在mongo中使用相同的名称空间，则可以连接use :mongod 0.app_name:27017，mongod 1.app_name:27017

对于您的每个应用程序

注: app_name=mongod

下面是一些例子：

apiVersion: v1
kind: Service
metadata:
  namespace: mongo-cluster
  name: mongo
  labels:
    app: mongo
    name: mongo
spec:
  type: ClusterIP
  ports:
  - port: 27017
    targetPort: 27017
  selector:
    app: mongo
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  namespace: mongo-cluster
  name: mongo
spec:
  serviceName: "mongo"
  replicas: 3
  selector:
    matchLabels:
      app: mongo
  template:
    metadata:
      labels:
        app: mongo
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: mongo
        image: mongo
        command:
          - "numactl"
          - "--interleave=all"
          - "mongod"
          - "--replSet"
          - "MainSetRep"
          - "--bind_ip"
          - "0.0.0.0"
          - "--auth"
          - "--clusterAuthMode" 
          - "keyFile" 
          - "--keyFile" 
          - "/etc/secrets-volume/mongodb-keyfile" 
          - "--setParameter" 
          - "authenticationMechanisms=SCRAM-SHA-1"
        ports:
          - containerPort: 27017
        volumeMounts:
          - name: data
            mountPath: /data/db
        volumeMounts:
          - name: mongo-key
            mountPath: "/etc/secrets-volume"
            readOnly: true
      volumes:
      - name: mongo-key
        secret:
          defaultMode: 0400
          secretName: mongo-key
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      storageClassName: openebs-hostpath
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 20G