未为带docker的elasticsearch集群分配分片

Question

这是我的合成文件，escp是我的docker图像文件

elasticsearch_master:
    #image: elasticsearch:latest
    image: escp
    command: "elasticsearch \
      -Des.cluster.name=dcluster \
      -Des.node.name=esmaster \
      -Des.node.master=true \
      -Des.node.data=true \    
      -Des.node.client=false \
      -Des.discovery.zen.minimum_master_nodes=1"

    volumes:
      - "${PWD}/es/config:/usr/share/elasticsearch/config"
      - "${PWD}/esdata/node:/usr/share/elasticsearch/data"
      - "${PWD}/es/plugins:/usr/share/elasticsearch/plugins"      
    environment:
       - ES_HEAP_SIZE=512m
    ports:
      - "9200:9200"
      - "9300:9300"

elasticsearch1:
    #image: elasticsearch:latest
    image: escp
    command: "elasticsearch \ 
      -Des.cluster.name=dcluster \
      -Des.node.name=esnode1 \
      -Des.node.data=true \
      -Des.node.client=false \
      -Des.node.master=false \
      -Des.discovery.zen.minimum_master_nodes=1 \
      -Des.discovery.zen.ping.unicast.hosts=elasticsearch_master"
    links:
      - elasticsearch_master
    volumes:
      - "${PWD}/es/config:/usr/share/elasticsearch/config"
      - "${PWD}/esdata/node1:/usr/share/elasticsearch/data"
      - "${PWD}/es/plugins:/usr/share/elasticsearch/plugins"
    environment:
       - ES_HEAP_SIZE=512m

elasticsearch2:
    #image: elasticsearch:latest
    image: escp
    command: "elasticsearch \
      -Des.cluster.name=dcluster \
      -Des.node.name=esnode2 \
      -Des.node.data=true \
      -Des.node.client=false \
      -Des.node.master=false \
      -Des.discovery.zen.minimum_master_nodes=1 \
      -Des.discovery.zen.ping.unicast.hosts=elasticsearch_master"
    links:
      - elasticsearch_master
    volumes:
      - "${PWD}/es/config:/usr/share/elasticsearch/config"
      - "${PWD}/esdata/node2:/usr/share/elasticsearch/data"
      - "${PWD}/es/plugins:/usr/share/elasticsearch/plugins"
    environment:
       - ES_HEAP_SIZE=512m

这是配置文件

index.number_of_shards: 1
index.number_of_replicas: 0
network.host: 0.0.0.0

运行后

           Name                         Command               State                       Ports
--------------------------------------------------------------------------------------------------------------------
est_elasticsearch1_1         /docker-entrypoint.sh elas ...   Up      9200/tcp, 9300/tcp
est_elasticsearch2_1         /docker-entrypoint.sh elas ...   Up      9200/tcp, 9300/tcp
est_elasticsearch_master_1   /docker-entrypoint.sh elas ...   Up      0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp

但当我创建新的索引时，会显示未分配...

curl -s '192.168.99.100:9200/_cluster/health?pretty'
{
  "cluster_name" : "dcluster",
  "status" : "red",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 1,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 0.0
}

检查节点

curl -s '192.168.99.100:9200/_cat/nodes?v'
host       ip         heap.percent ram.percent load node.role master name
172.17.0.2 172.17.0.2           13          33 0.00 d         *      esmaster
172.17.0.3 172.17.0.3           16          33 0.00 d         -      esnode1
172.17.0.4 172.17.0.4           13          33 0.00 d         -      esnode2

检查碎片

curl -s '192.168.99.100:9200/_cat/shards'
abcq 0 p UNASSIGNED

检查分配

curl -s '192.168.99.100:9200/_cat/allocation?v'
shards disk.indices disk.used disk.avail disk.total disk.percent host       ip         node
     0           0b   223.4gb      9.5gb    232.9gb           95 172.17.0.4 172.17.0.4 esnode2
     0           0b   223.4gb      9.5gb    232.9gb           95 172.17.0.2 172.17.0.2 esmaster
     0           0b   223.4gb      9.5gb    232.9gb           95 172.17.0.3 172.17.0.3 esnode1
     1                                                                                 UNASSIGNED

检查设置

curl 'http://192.168.99.100:9200/_cluster/settings?pretty'
{
  "persistent" : { },
  "transient" : { }
}

已启用重新路由

curl 'http://192.168.99.100:9200/_cluster/settings?pretty'
{
  "persistent" : { },
  "transient" : {
    "cluster" : {
      "routing" : {
        "allocation" : {
          "enable" : "true"
        }
      }
    }
  }
}

重新路由索引abcq

    curl -XPOST http://192.168.99.100:9200/_cluster/reroute?pretty -d '{ 
    "commands" : [ 
        { 
            "allocate" : {
                "index" : "abcq",
                "shard" : 0, 
                "node" : "esnode2", 
                "allow_primary" : true 
            } 
        } 
    ] 
}'

获取错误如下

{
  "error" : {
    "root_cause" : [ {
      "type" : "illegal_argument_exception",
      "reason" : "[allocate] allocation of [abcq][0] on node {esnode2}{Pisl95VUSPmZa3Ga_e3sDA}{172.17.0.4}{172.17.0.4:9300}{master=false} is not allowed, reason: [YES(shard is primary)][YES(no allocation awareness enabled)][NO(more than allowed [90.0%] used disk on node, free: [4.078553722498398%])][YES(allocation disabling is ignored)][YES(primary shard can be allocated anywhere)][YES(node passes include/exclude/require filters)][YES(shard is not allocated to same node or host)][YES(total shard limit disabled: [index: -1, cluster: -1] <= 0)][YES(allocation disabling is ignored)][YES(no snapshots are currently running)][YES(below primary recovery limit of [4])]"
    } ],
    "type" : "illegal_argument_exception",
    "reason" : "[allocate] allocation of [abcq][0] on node {esnode2}{Pisl95VUSPmZa3Ga_e3sDA}{172.17.0.4}{172.17.0.4:9300}{master=false} is not allowed, reason: [YES(shard is primary)][YES(no allocation awareness enabled)][NO(more than allowed [90.0%] used disk on node, free: [4.078553722498398%])][YES(allocation disabling is ignored)][YES(primary shard can be allocated anywhere)][YES(node passes include/exclude/require filters)][YES(shard is not allocated to same node or host)][YES(total shard limit disabled: [index: -1, cluster: -1] <= 0)][YES(allocation disabling is ignored)][YES(no snapshots are currently running)][YES(below primary recovery limit of [4])]"
  },
  "status" : 400
}

为什么我创建新的索引没有被分配，有人可以帮助我吗？谢谢。

Answer 1

这里的人是如何解决问题的。

more than allowed [90.0%] used disk on node

这意味着我的磁盘已经满了，没有太多的空间来分配分片。

shards disk.indices disk.used disk.avail disk.total disk.percent host       ip         node
     0           0b   223.4gb      9.5gb    232.9gb           95 172.17.0.4 172.17.0.4 esnode2

禁用磁盘检查或将其设置得更低

curl -XPUT localhost:9200/_cluster/settings -d '{
    "transient" : {
        "cluster.routing.allocation.disk.threshold_enabled" : false
    }
}'
curl -XPUT http://192.168.99.100:9200/_cluster/settings -d '
{
    "transient" : {
        "cluster.routing.allocation.disk.watermark.low": "10%",
        "cluster.routing.allocation.disk.watermark.high": "10gb",
        "cluster.info.update.interval": "1m"
    }
}'

希望这能对其他人有所帮助，更多细节可以在这里查看。https://www.elastic.co/guide/en/elasticsearch/reference/current/disk-allocator.html