使用TLS的CoreOS etcd :设备失败，但在日志中没有任何原因

Question

我正在尝试使用TLS在我的CoreOS集群设置上获取etcd ...度过了一段地狱般的时光。

我查看了不同的指南，生成了客户端和同级证书和密钥

etcd启动失败，我在journalctl中得到的信息如下(IP和token混淆)：

Dec 16 00:05:12 coreos-123.123.123.123 systemd[1]: Starting etcd2...
-- Subject: Unit etcd2.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit etcd2.service has begun starting up.
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=http://123.123.123.123:2379
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_CERT_FILE=/etc/ssl/etcd/etcd-client123.123.123.123.cert.pem
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_CLIENT_CERT_AUTH=true
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd2
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_DISCOVERY=https://discovery.etcd.io/xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=http://123.123.123.123:2380
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_KEY_FILE=/etc/ssl/etcd/private/etcd-client123.123.123.123.key.pem
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379,http://0.0.0.0:4001
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_LISTEN_PEER_URLS=http://123.123.123.123:2380,http://123.123.123.123:7001
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_NAME=yyyyyyyyyyyyyyyyyyyyyyyyyy
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_PEER_CERT_FILE=/etc/ssl/etcd/etcd-peer123.123.123.123.cert.pem
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_PEER_CLIENT_CERT_AUTH=true
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_PEER_KEY_FILE=/etc/ssl/etcd/private/etcd-peer123.123.123.123.key.pem
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_PEER_TRUSTED_CA_FILE=/etc/ssl/certs/ca-chain.cert.pem
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: recognized and used environment variable ETCD_TRUSTED_CA_FILE=/etc/ssl/certs/ca-chain.cert.pem
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: etcd Version: 2.2.0
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: Git SHA: e4561dd
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: Go Version: go1.4.2
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: Go OS/Arch: linux/amd64
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: setting maximum number of CPUs to 1, total number of available CPUs is 4
Dec 16 00:05:12 coreos-123.123.123.123 etcd2[822]: the server is already initialized as member before, starting as etcd member...
Dec 16 00:05:12 coreos-123.123.123.123 systemd[1]: etcd2.service: Main process exited, code=exited, status=1/FAILURE
Dec 16 00:05:12 coreos-123.123.123.123 systemd[1]: Failed to start etcd2.
-- Subject: Unit etcd2.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit etcd2.service has failed.
--
-- The result is failed.
Dec 16 00:05:12 coreos-123.123.123.123 systemd[1]: etcd2.service: Unit entered failed state.
Dec 16 00:05:12 coreos-123.123.123.123 systemd[1]: etcd2.service: Failed with result 'exit-code'.

我将证书和密钥放在正确的文件夹中。我非常确定权限是正常的。证书具有针对clientAuth、serverAuth (用于对等证书)和clientAuth(用于客户端)以及具有节点IP的SAN的扩展。

客户端证书数据：

Exponent: 65537 (0x10001)
X509v3 extensions:
    X509v3 Basic Constraints:
        CA:FALSE
    Netscape Cert Type:
        SSL Client, S/MIME
    Netscape Comment:
        OpenSSL Generated Client Certificate
    X509v3 Subject Key Identifier:

    X509v3 Authority Key Identifier:
        keyid:

    X509v3 Key Usage: critical
        Digital Signature, Non Repudiation, Key Encipherment
    X509v3 Extended Key Usage:
        TLS Web Client Authentication, E-mail Protection
    X509v3 Subject Alternative Name:
        IP Address:127.0.0.1, IP Address:123.123.123.123

同级证书数据：

X509v3 extensions:
            X509v3 Basic Constraints:
                CA:FALSE
            Netscape Cert Type:
                SSL Server
            Netscape Comment:
                OpenSSL Generated Server Certificate
            X509v3 Subject Key Identifier:

            X509v3 Authority Key Identifier:
                keyid:

            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Subject Alternative Name:
                IP Address:127.0.0.1, IP Address:123.123.123.123

我还遗漏了什么？此日志中没有任何内容来解释失败。

我的目标是在公共云上为客户端和对等点提供TLS身份验证。PS:它在没有TLS的情况下工作得很好。我只添加了证书和8个TLS标志：

# client flags
    trusted-ca-file: /etc/ssl/certs/ca-chain.cert.pem
    cert-file: /etc/ssl/etcd/etcd-client$public_ipv4.cert.pem
    key-file: /etc/ssl/etcd/private/etcd-client$public_ipv4.key.pem
    client-cert-auth: true

    # peer flags
    peer-trusted-ca-file: /etc/ssl/certs/ca-chain.cert.pem
    peer-cert-file: /etc/ssl/etcd/etcd-peer$public_ipv4.cert.pem
    peer-key-file: /etc/ssl/etcd/private/etcd-peer$public_ipv4.key.pem
    peer-client-cert-auth: true

由于日志中显示了该IP，因此$public_ipv4标签明显被正确转换

我只是不知道这里的问题是什么，因为日志没有说太多。

有没有办法给我指明正确的方向？

谢谢

Answer 1

由于上游systemd错误，journald在其进程退出时可能会遗漏最后几行日志。如果journalctl告诉您etcd停止而没有出现致命或紧急消息，您可以尝试使用sudo journalctl -f -t etcd2来获取完整的日志。

一旦你有了完整的日志，它应该会告诉你etcd失败的原因。