共享Mongodb摊位随机

Question

我已经使用kuberenetesHassed切分设置了共享MongoDB集群，我首先创建了配置服务器Replicaset，然后创建了2个shard副本集。最后创建了mongos来连接到切分群集。

我按照下面的链接设置切分MongoDB，单击https://docs.mongodb.com/manual/tutorial/deploy-sharded-cluster-hashed-sharding/

在创建mongos之后，我已经为数据库启用了切分，并使用散列切分策略对集合进行了切分。

在完成所有这些设置之后，我能够连接到mongos，并将一些数据添加到数据库中的一些集合中，并能够检查数据在不同碎片之间的分布情况。

我面临的问题是，当试图从我的java spring引导项目访问mongodb时，一旦为特定查询建立了连接，连接就会停止randomly.But，如果我再次向mongodb提出请求，该特定的查询不会在下几个空闲时间停止，它将再次开始停止。

注意: MongoDB托管在"DS2 v2“VM中，这个集群有4个节点1用于配置服务器，2个用于碎片，1个用于mongos

在其中一个链接中，他们要求为所有集合设置适当的碎片键，这将对mongodb.There的性能产生影响。在选择正确的碎片键之前，需要考虑以下几点:在选择shard键之前，我考虑了所有这些因素。

我遇到的另一个解决方案是设置ShardingTaskExecutorPoolMaxConnecting并限制mongos节点将连接添加到连接池的速率，我尝试将其设置为20、5100、150，但没有一个解决了我面临的延迟问题。这是链接-单击https://jira.mongodb.org/browse/SERVER-29237

我试着调整其他参数，如ShardingTaskExecutorPoolMinSize和taskExecutorPoolSize.Even，但这并没有解决拖延问题。

我还设置

将wiredTigerCacheSizeGB从0.25增加到2。这也会对拖延问题产生影响。

1) mongodb配置服务器的服务和部署YAML文件为：

apiVersion: v1
items:
- apiVersion: v1
  kind: Service
  metadata:
    annotations:
      kompose.cmd: kompose convert -d -f docker-compose.yml -o azure-deployment.yaml
      kompose.version: 1.12.0 (0ab07be)
    creationTimestamp: null
    labels:
      io.kompose.service: mongo-conf-service
    name: mongo-conf-service
  spec:
    type: LoadBalancer
    ports:
    - name: "27017"
      port: 27017
      targetPort: 27017
    selector:
      io.kompose.service: mongo-conf-service
  status:
    loadBalancer: {}
- apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    annotations:
      kompose.cmd: kompose convert -d -f docker-compose.yml -o azure-deployment.yaml
      kompose.version: 1.12.0 (0ab07be)
    creationTimestamp: null
    labels:
      io.kompose.service: mongo-conf-service
    name: mongo-conf-service
  spec:
    replicas: 1
    strategy: {}
    template:
      metadata:
        creationTimestamp: null
        labels:
          io.kompose.service: mongo-conf-service
      spec:
        containers:
        - env:
          - name: MONGO_INITDB_ROOT_USERNAME
            value: #Username
          - name: MONGO_INITDB_ROOT_PASSWORD
            value: #Password
          command:
          - "mongod"
          - "--storageEngine"
          - "wiredTiger"
          - "--port"
          - "27017"
          - "--bind_ip"
          - "0.0.0.0"
          - "--wiredTigerCacheSizeGB"
          - "2"
          - "--configsvr"
          - "--replSet"
          - "ConfigDBRepSet"
          image: #MongoImageName
          name: mongo-conf-service
          ports:
          - containerPort: 27017
          resources: {}
          volumeMounts:
          - name: mongo-conf
            mountPath: /data/db
        restartPolicy: Always
        volumes:
          - name: mongo-conf
            persistentVolumeClaim:
              claimName: mongo-conf

2)为Shard mongodb提供服务和部署的YAML文件是：

apiVersion: v1
items:
- apiVersion: v1
  kind: Service
  metadata:
    annotations:
      kompose.cmd: kompose convert -d -f docker-compose.yml -o azure-deployment.yaml
      kompose.version: 1.12.0 (0ab07be)
    creationTimestamp: null
    labels:
      io.kompose.service: mongo-shard
    name: mongo-shard
  spec:
    type: LoadBalancer
    ports:
    - name: "27017"
      port: 27017
      targetPort: 27017
    selector:
      io.kompose.service: mongo-shard
  status:
    loadBalancer: {}
- apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    annotations:
      kompose.cmd: kompose convert -d -f docker-compose.yml -o azure-deployment.yaml
      kompose.version: 1.12.0 (0ab07be)
    creationTimestamp: null
    labels:
      io.kompose.service: mongo-shard
    name: mongo-shard
  spec:
    replicas: 1
    strategy: {}
    template:
      metadata:
        creationTimestamp: null
        labels:
          io.kompose.service: mongo-shard
      spec:
        containers:
        - env:
          - name: MONGO_INITDB_ROOT_USERNAME
            value: #Username
          - name: MONGO_INITDB_ROOT_PASSWORD
            value: #Password
          command:
          - "mongod"
          - "--storageEngine"
          - "wiredTiger"
          - "--port"
          - "27017"
          - "--bind_ip"
          - "0.0.0.0"
          - "--wiredTigerCacheSizeGB"
          - "2"
          - "--shardsvr"
          - "--replSet"
          - "Shard1RepSet"
          image: #MongoImage
          name: mongo-shard
          ports:
          - containerPort: 27017
          resources: {}

3) mongos服务器的YAML文件：

apiVersion: v1
items:
- apiVersion: v1
  kind: Service
  metadata:
    annotations:
      kompose.cmd: kompose convert -d -f docker-compose.yml -o azure-deployment.yaml
      kompose.version: 1.12.0 (0ab07be)
    creationTimestamp: null
    labels:
      io.kompose.service: mongos-service
    name: mongos-service
  spec:
    type: LoadBalancer
    ports:
    - name: "27017"
      port: 27017
      targetPort: 27017
    selector:
      io.kompose.service: mongos-service
  status:
    loadBalancer: {}
- apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    annotations:
      kompose.cmd: kompose convert -d -f docker-compose.yml -o azure-deployment.yaml
      kompose.version: 1.12.0 (0ab07be)
    creationTimestamp: null
    labels:
      io.kompose.service: mongos-service
    name: mongos-service
  spec:
    replicas: 1
    strategy: {}
    template:
      metadata:
        creationTimestamp: null
        labels:
          io.kompose.service: mongos-service
      spec:
        containers:
        - env:
          - name: MONGO_INITDB_ROOT_USERNAME
            value: #USername
          - name: MONGO_INITDB_ROOT_PASSWORD
            value: #Password
          command:
            - "numactl"
            - "--interleave=all"
            - "mongos"
            - "--port"
            - "27017"
            - "--bind_ip"
            - "0.0.0.0"
            - "--configdb"
            - "ConfigDBRepSet/mongo-conf-service:27017"
          image: #MongoImageName
          name: mongos-service
          ports:
          - containerPort: 27017
          resources: {}

mongos服务器的日志是：

2019-08-05T05:27:52.942+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:5058 #308807 (79 connections now open)
2019-08-05T05:27:52.964+0000 I ACCESS   [conn308807] Successfully authenticated as principal Assist_Random_Workspace on Random_Workspace from client 10.0.0.0:5058
2019-08-05T05:27:54.267+0000 I NETWORK  [worker-3] end connection 10.0.0.0:52954 (78 connections now open)
2019-08-05T05:27:54.269+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:52988 #308808 (79 connections now open)
2019-08-05T05:27:54.275+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:7174 #308809 (80 connections now open)
2019-08-05T05:27:54.279+0000 I ACCESS   [conn308809] SASL SCRAM-SHA-1 authentication failed for Assist_Refactored_Code_DB on Refactored_Code_DB from client 10.0.0.:7174 ; UserNotFound: User "Assist_Refactored_Code_DB@Refactored_Code_DB" not found
2019-08-05T05:27:54.281+0000 I NETWORK  [worker-1] end connection 10.0.0.5:7174 (79 connections now open)
2019-08-05T05:27:54.342+0000 I NETWORK  [worker-1] end connection 10.0.0.6:57391 (78 connections now open)
2019-08-05T05:27:54.343+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:57527 #308810 (79 connections now open)
2019-08-05T05:27:55.080+0000 I NETWORK  [worker-3] end connection 10.0.0.0:56021 (78 connections now open)
2019-08-05T05:27:55.081+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:56057 #308811 (79 connections now open)
2019-08-05T05:27:56.054+0000 I NETWORK  [worker-1] end connection 10.0.0.0:59137 (78 connections now open)
2019-08-05T05:27:56.055+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:59184 #308812 (79 connections now open)
2019-08-05T05:27:59.268+0000 I NETWORK  [worker-1] end connection 10.0.0.5:52988 (78 connections now open)
2019-08-05T05:27:59.270+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:53047 #308813 (79 connections now open)
2019-08-05T05:27:59.343+0000 I NETWORK  [worker-3] end connection 10.0.0.6:57527 (78 connections now open)
2019-08-05T05:27:59.344+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:57672 #308814 (79 connections now open)
2019-08-05T05:28:00.080+0000 I NETWORK  [worker-3] end connection 10.0.1.1:56057 (78 connections now open)
2019-08-05T05:28:00.081+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:56116 #308815 (79 connections now open)
2019-08-05T05:28:01.054+0000 I NETWORK  [worker-3] end connection 10.0.0.0:59184 (78 connections now open)
2019-08-05T05:28:01.058+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:59225 #308816 (79 connections now open)
2019-08-05T05:28:01.763+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:7173 #308817 (80 connections now open)
2019-08-05T05:28:01.768+0000 I ACCESS   [conn308817] SASL SCRAM-SHA-1 authentication failed for Assist_Sharded_Database on Sharded_Database from client 10.0.0.0:7173 ; UserNotFound: User "Assist_Sharded_Database@Sharded_Database" not found
2019-08-05T05:28:01.770+0000 I NETWORK  [worker-3] end connection 10.0.0.0:7173 (79 connections now open)
2019-08-05T05:28:04.271+0000 I NETWORK  [worker-3] end connection 10.0.0.0:53047 (78 connections now open)
2019-08-05T05:28:04.272+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:53083 #308818 (79 connections now open)
2019-08-05T05:28:04.283+0000 I NETWORK  [listener] connection accepted from 10.0.0.0:7105 #308819 (80 connections now open)
2019-08-05T05:28:04.287+0000 I ACCESS   [conn308819] SASL SCRAM-SHA-1 authentication failed for Assist_Refactored_Code_DB on Refactored_Code_DB from client 10.0.0.0:7105 ; UserNotFound: User "Assist_Refactored_Code_DB@Refactored_Code_DB" not found

连接到MongoDB的Java块是-

注意:下面的代码支持对每个请求中的一个参数进行多租户的MongoDB at Databaselevel.Based，我们将确定从哪个数据库进行查询。

对于独立的MongoDB实例，下面的代码将很好地工作。

1)应用财产

mongodb.uri=${mongoURI:mongodb://username:password@IPaddress:portNumber}
mongodb.defaultDatabaseName=assist
assist-server-address1 = IpAddress

2)弹簧引导应用

@SpringBootApplication
@ServletComponentScan
public class ServiceApplication extends RepositoryRestConfigurerAdapter {
    public static void main(String[] args) {
        SpringApplication.run(ServiceApplication.class, args);
    }

    @Autowired
    public MongoDBCredentials mongoDBCredentials;

    @Bean
    public MongoTemplate mongoTemplate() {
        return new MongoTenantTemplate(
                new SimpleMongoDbFactory(new MongoClient(new MongoClientURI(mongoDBCredentials.getUri())),
                        mongoDBCredentials.getDefaultDatabaseName()));
    }

}

3)MongoTemplate ->这是用来建立connection.We的，实现了多租户connection.We(基于请求的一个参数连接到多个数据库)。

public class MongoTenantTemplate extends MongoTemplate {
    private static Map tenantTemplates = new HashMap<>();

    @Value("${assist-server-address1}")
    public String ServerAddress1;

    @Value("${spring.data.mongodb.username}")
    public String ServerUsername;

    @Value("${spring.data.mongodb.password}")
    public String ServerPassword;

    @Value("${spring.data.mongodb.database}")
    public String ServerDbName;

    @Value("${assist-current-environment}")
    public String currentEnv;

    @Autowired
    public MongoDBCredentials mongoDBCredentials;

    @Autowired
    WorkspacesRepository workspaceRepository;

    private static final Logger LOG = LoggerFactory.getLogger(MongoTenantTemplate.class);
    Marker marker;

    public MongoTenantTemplate(MongoDbFactory mongoDbFactory) {
        super(mongoDbFactory);
        tenantTemplates.put(mongoDbFactory.getDb().getName(), new MongoTemplate(mongoDbFactory));
    }

    protected MongoTemplate getTenantMongoTemplate(String tenant) {

        MongoTemplate mongoTemplate = tenantTemplates.computeIfAbsent(tenant, k -> null);
        LOG.info(marker, "Tenant is (MongoDBCredentials) : {}",tenant);
        try {
            if (mongoTemplate == null) {
                MongoCredential mongoCredential;
                // Username,databaseName,password
                if (tenant == ServerDbName) {
                    mongoCredential = MongoCredential.createCredential(ServerUsername, ServerDbName,
                            ServerPassword.toCharArray());
                } else {
                    Workspaces workspace = workspaceRepository.findByDbName(tenant);
                    mongoCredential = MongoCredential.createCredential(workspace.getDbUserName(), workspace.getDbName(),
                            workspace.getDbPassword().toCharArray());
                }
                ServerAddress address1 = new ServerAddress(ServerAddress1, port);

                List serverAddressList = new ArrayList<>();
                serverAddressList.add(address1);

                SimpleMongoDbFactory mongoDbFactory = new SimpleMongoDbFactory(
                        new MongoClient(serverAddressList, Arrays.asList(mongoCredential)), tenant);

                mongoTemplate = new MongoTemplate(mongoDbFactory);
            }
            else {
            }
        } catch (Exception e) {
            tenantTemplates.remove(tenant);
        }
        return mongoTemplate;
    }
...

在上面的日志中，对Assist_Refactored_Code_DB的身份验证(这个数据库不是由我创建的)出现了一个错误，.Im不确定为什么这个身份验证失败，以及在哪个mongo中用户名和密码应该是mentioned.And，我也不确定这是否是延迟的原因之一。这是我能在mongos.All中找到的唯一错误日志，配置服务器和shard中的其他日志没有任何错误。

我希望切分的mongodb不会在任何时候停止工作，并与独立的mongodb类似。

有谁能引导我解决被分割的mongodb问题呢？

Answer 1

您的mongo日志引发身份验证错误：

2019-08-05T05:27:54.279+0000 I ACCESS   [conn308809] SASL SCRAM-SHA-1 authentication failed for Assist_Refactored_Code_DB on Refactored_Code_DB from client 10.0.0.:7174 ; UserNotFound: User "Assist_Refactored_Code_DB@Refactored_Code_DB" not found

请先检查用户"Assist_Refactored_Code_DB@Refactored_Code_DB“是否在用户集合中创建。

谢谢!

-Anban