如何从CoreOS上停靠映像中的快照恢复etcd群集？

Question

我有一个Kubernetes集群(v1.5.6)，在vmware上有3个节点etcd集群(etcd版本3.1.5)。这个etcd节点运行在vmware上的coreos上的三个码头容器(三个主机上)中。

我尝试使用以下解决方案备份etcd：

docker run --rm --net=host -v /tmp:/etcd_backup -e ETCDCTL_API=3 quay.io/coreos/etcd:v3.1.5 etcdctl --endpoints=[1.1.1.1:2379,2.2.2.2:2379,3.3.3.3:2379] snapshot save etcd_backup/snapshot.db

备份已成功完成。

我想在另一个vmware环境中从零创建这个kubernetes集群，但是我需要从快照恢复etcd。

到目前为止，我还没有找到在码头集装箱中使用etcd的正确解决方案。

我试图用以下方法进行恢复，但不幸的是，我没有成功。

首先，在运行以下命令之后，我创建了一个新的etcd节点：

docker run --rm --net=host -v /tmp/etcd_bak:/etcd_backup -e ETCDCTL_API=3 registry:5000/quay.io/coreos/etcd:v3.1.5 etcdctl snapshot restore etcd_backup/snapshot.db --name etcd0 --initial-cluster etcd0=http://etcd0:2380,etcd1=http://etcd1:2380,etcd2=http://etcd2:2380 --initial-cluster-token etcd-cluster-1 --initial-advertise-peer-urls http://etcd0:2380

结果：

2018-06-04 09:25:52.314747 I | etcdserver/membership: added member 7ff5c9c6942f82e [http://etcd0:2380] to cluster 5d1b637f4b7740d5
2018-06-04 09:25:52.314940 I | etcdserver/membership: added member 91b417e7701c2eeb [http://etcd2:2380] to cluster 5d1b637f4b7740d5
2018-06-04 09:25:52.315096 I | etcdserver/membership: added member faeb78734ee4a93d [http://etcd1:2380] to cluster 5d1b637f4b7740d5

不幸的是，什么都没发生。

恢复etcd备份的好解决方案是什么？

如何创建空的etcd集群/节点，以及如何恢复快照？

Answer 1

根据Etcd 灾后恢复文档，需要将快照中的所有三个etcd节点还原为您的命令，然后使用以下命令运行三个节点：

etcd \
  --name m1 \
  --listen-client-urls http://host1:2379 \
  --advertise-client-urls http://host1:2379 \
  --listen-peer-urls http://host1:2380 &

此外，还可以从图像中提取etcdctl，如下所示：

docker run --rm -v /opt/bin:/opt/bin registry:5000/quay.io/coreos/etcd:v3.1.5 cp /usr/local/bin/etcdctl /opt/bin

然后使用etcdctl恢复快照：

# ETCDCTL_API=3 ./etcdctl snapshot restore snapshot.db \
  --name m1 \
  --initial-cluster m1=http://host1:2380,m2=http://host2:2380,m3=http://host3:2380 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-advertise-peer-urls http://host1:2380 \
  --data-dir /var/lib/etcd

这将将快照还原到/var/lib/etcd目录。然后用docker启动etcd，不要忘记将/var/lib/etcd挂载到容器中，并指定--数据-dir到容器。

Answer 2

kubernetes中的Ectd运行在Docker容器中，下面是我为恢复集群所做的工作：

• 检索Etcd集群流星 码头检查etcd1 你会发现如下所示： "Binds"："/etc/ssl/certs:/etc/ssl/certs:ro"，"/etc/ssl/etcd/ssl:/etc/ssl/etcd/ssl:ro"，"/var/lib/etcd:/var/lib/etcd:rw“，"Env"："ETCD_DATA_DIR=/var/lib/etcd"，"ETCD_ADVERTISE_CLIENT_URLS=https://172.16.60.1:2379"，"ETCD_INITIAL_ADVERTISE_PEER_URLS=https://172.16.60.1:2380"，"ETCD_INITIAL_CLUSTER_STATE=existing"，"ETCD_METRICS=basic"，"ETCD_LISTEN_CLIENT_URLS=https://172.16.60.1:2379，https://127.0.0.1:2379"，"ETCD_ELECTION_TIMEOUT=5000“、"ETCD_HEARTBEAT_INTERVAL=250”、"ETCD_INITIAL_CLUSTER_TOKEN=k8s_etcd“、"ETCD_LISTEN_PEER_URLS=https://172.16.60.1:2380"，"ETCD_NAME=etcd1”、"ETCD_PROXY=off“、"ETCD_INITIAL_CLUSTER=etcd1=https://172.16.60.1:2380，etcd2=https://172.16.60.2:2380，Https://172.16.60.2:2380“，"ETCD_AUTO_COMPACTION_RETENTION=8"，"ETCD_TRUSTED_CA_FILE=/etc/ssl/etcd/ssl/ca.pem"，"ETCD_CERT_FILE=/etc/ssl/etcd/ssl/member-node01.pem"，"ETCD_KEY_FILE=/etc/ssl/etcd/ssl/member-node01-key.pem"，"ETCD_PEER_TRUSTED_CA_FILE=/etc/ssl/etcd/ssl/ca.pem"，"ETCD_PEER_CERT_FILE=/etc/ssl/etcd/ssl/member-node01.pem"，"ETCD_PEER_KEY_FILE=/etc/ssl/etcd/ssl/member-node01-key.pem"，"ETCD_PEER_CLIENT_CERT_AUTH=true"，"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin“，"Cmd"："/usr/local/bin/etcd”
• 将etcd快照数据库复制到其他etcd节点 scp snapshotdb_20180913节点02:/root/ scp snapshotdb_20180913节点03:/snapshotdb_20180913/
• 使用原始的infoetcd1停靠点停止etcd1 rm /var/lib/etcd ETCDCTL_API=3 etcdctl快照恢复snapshotdb_20180913 \-cacert /etc/ssl/etcd/ssl/ca.pem \-cert/etc/ssl/etcd/ssl/ etcd1 -node01.pem\-key/etc/ssl/rm/ssl/ETCDCTL_API=3成员-node01-key.pem\-名称etcd1\-初始群集etcd1=https://node01:2380,etcd2=https://node02:2380,etcd3=https://node03:2380 \-初始群集标记- k8s_etcd \-初始-群集etcd1=https://node01:2380,etcd2=https://node02:2380,etcd3=https://node03:2380 \-初始-群集-令牌k8s_etcd \-初始-广告-对等-urls https://node02:2380 \-数据-dir /var/lib/etcd # etcd3码头停止etcd3 rm -rf /var/lib/etcd ETCDCTL_API=3 etcdctl快照恢复snapshotdb_20180913 \-cacert /etc/ssl/etcd/ssl/ca.pem \-cert/etc/ssl/etcd/ssl成员-node03.pem\-密钥/etc/ssl/etcd/ssl/member node03-key.pem\- etcd3 \-初始化- etcd1=https://node01:2380,etcd2=https://node02:2380,etcd3=https://node03:2380 \-初始-群集-令牌k8s_etcd \-初始-公告-对等urls https://node03:2380 \-数据-dir /var/lib/etcd
• 启动容器并检查群集状态 cd /etc/ssl/etcd/ssl etcdctl \-终结点=https://node01:2379\-ca-file=./ca.pem\-cert-file=./ member node01.pem\--key-file=./emb-node01-key.pem\key.pem列表