向现有elasticsearch节点添加第二个主节点的正确方法

Question

我有两台相同的服务器，我在其中一台上设置了弹性搜索，并向其中导入了大约200m条记录。我现在希望另一台服务器有相同的一组记录，并且两台服务器成对工作，以防其中一台服务器宕机，我可以快速从集群中删除另一台服务器，它将正常工作，不会插入新数据，它只是读取。我跟踪了我在网上找到的信息，并设置了我的主机来找到对方。在我看到的正在复制的分片的状态中，索引似乎没有传输到其他服务器，但它们在数据目录中的大小从来没有增加过。

我已经将其用作指针。

How to add a new node to my Elasticsearch cluster

我的问题是，我是否可以通过scp将超过300 do的数据复制到数据目录，或者elasticsearch是否需要这样做？

调试信息

curl 192.168.1.1:9200/_cat/recovery?v
index  shard time   type       stage source_host target_host repository snapshot files files_percent bytes       bytes_percent total_files total_bytes translog translog_percent total_translog
pelias 0     248094 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        76          13092952848 0        100.0%           0
pelias 0     333911 relocation index 192.168.1.2 192.168.1.1 n/a        n/a      76    76.3%         13092952848 9.6%          76          13092952848 0        100.0%           0
pelias 1     248053 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        70          12924273847 0        100.0%           0
pelias 1     333915 relocation index 192.168.1.2 192.168.1.1 n/a        n/a      70    98.6%         12924273847 98.0%         70          12924273847 0        100.0%           0
pelias 2     246409 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        88          13122252645 0        100.0%           0
pelias 3     247434 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        89          13020268507 0        100.0%           0
pelias 4     247109 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        91          13173644369 0        100.0%           0
pelias 5     247574 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        85          13133234395 0        100.0%           0
pelias 6     245236 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        82          12882617780 0        100.0%           0
pelias 7     246530 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        86          13098807983 0        100.0%           0
pelias 8     245089 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        73          12963045661 0        100.0%           0
pelias 9     245149 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        94          13072471481 0        100.0%           0
pelias 10    248994 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        85          12950316547 0        100.0%           0
pelias 11    247667 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        73          13329893241 0        100.0%           0
pelias 12    248055 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        79          12930959145 0        100.0%           0
pelias 13    249255 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        106         11586141290 0        100.0%           0
pelias 14    247759 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        82          15107208138 0        100.0%           0
pelias 15    247888 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        79          13112198101 0        100.0%           0
pelias 16    247675 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        91          14672189956 0        100.0%           0
pelias 17    247205 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        73          13048452056 0        100.0%           0
pelias 18    246910 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        82          13095873096 0        100.0%           0
pelias 19    246325 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        70          15019530308 0        100.0%           0
pelias 20    247924 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        77          12898451788 0        100.0%           0
pelias 21    248663 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        76          13035468700 0        100.0%           0
pelias 22    249105 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        95          13045543380 0        100.0%           0
pelias 23    246119 store      done  192.168.1.2 192.168.1.2 n/a        n/a      0     100.0%        0           100.0%        79          13052787518 0        100.0%           0



     curl 192.168.1.1:9200/_cat/shards?v
index  shard prirep state          docs  store ip          node
pelias 22    p      STARTED    21505006 12.1gb 192.168.1.2 ronin-tile02
pelias 5     p      STARTED    21503012 12.2gb 192.168.1.2 ronin-tile02
pelias 2     p      STARTED    21499705 12.2gb 192.168.1.2 ronin-tile02
pelias 15    p      STARTED    21504176 12.2gb 192.168.1.2 ronin-tile02
pelias 4     p      STARTED    21512286 12.2gb 192.168.1.2 ronin-tile02
pelias 16    p      STARTED    21508361 13.6gb 192.168.1.2 ronin-tile02
pelias 20    p      STARTED    21499566   12gb 192.168.1.2 ronin-tile02
pelias 6     p      STARTED    21504264 11.9gb 192.168.1.2 ronin-tile02
pelias 14    p      STARTED    21511288   14gb 192.168.1.2 ronin-tile02
pelias 23    p      STARTED    21513431 12.1gb 192.168.1.2 ronin-tile02
pelias 13    p      STARTED    21505815 10.7gb 192.168.1.2 ronin-tile02
pelias 12    p      STARTED    21506998   12gb 192.168.1.2 ronin-tile02
pelias 10    p      STARTED    21511704   12gb 192.168.1.2 ronin-tile02
pelias 9     p      STARTED    21504736 12.1gb 192.168.1.2 ronin-tile02
pelias 1     p      RELOCATING 21506366   12gb 192.168.1.2 ronin-tile02 -> 192.168.1.1 PsmIinmpQ5myXE5oNdw4qA ronin-tile01
pelias 7     p      STARTED    21508186 12.1gb 192.168.1.2 ronin-tile02
pelias 21    p      STARTED    21511618 12.1gb 192.168.1.2 ronin-tile02
pelias 17    p      STARTED    21511014 12.1gb 192.168.1.2 ronin-tile02
pelias 18    p      STARTED    21512577 12.1gb 192.168.1.2 ronin-tile02
pelias 8     p      STARTED    21511765   12gb 192.168.1.2 ronin-tile02
pelias 11    p      STARTED    21504697 12.4gb 192.168.1.2 ronin-tile02
pelias 3     p      STARTED    21509465 12.1gb 192.168.1.2 ronin-tile02
pelias 19    p      STARTED    21503146 13.9gb 192.168.1.2 ronin-tile02
pelias 0     p      RELOCATING 21512876 12.1gb 192.168.1.2 ronin-tile02 -> 192.168.1.1 PsmIinmpQ5myXE5oNdw4qA ronin-tile01

  curl 192.168.1.1:9200/_cat/indices?v
health status index  pri rep docs.count docs.deleted store.size pri.store.size
green  open   pelias  24   0  516182058           89    295.5gb        295.5gb

Answer 1

为了在两个(或更多) Elasticsearch节点之间复制数据，您需要添加replicas。对于索引(pelias)和主机(192.168.1.1:9200)，可以使用以下curl命令添加单个复制副本：

curl -XPUT 'http://192.168.1.1:9200/pelias/_settings' -d '{
  "index": {
    "number_of_replicas": 1
  }
}'

顺便说一句，感谢您使用Pelias，我是核心维护者之一:)

我还维护了一组bash脚本来完成常见的Elasticsearch任务here。有一个专门针对adding replicas的脚本，您或其他人可能会发现它很有用。