证书对ingress.local有效，而不是gitlab.mydomain。

Question

我对Kubernetes很陌生。

• 我使用RKE (用于创建k8集群的牧场主工具)安装了一个新的Kubernetes集群。
• 我添加了gitlab图表(https://charts.gitlab.io/)并启动它。
• 在PersistentStorage等几个问题上，我设法解决了。

但是，我现在还停留在最后一个问题上：gitlab-runner的吊舱在以下日志中失败了：

ERROR: Registering runner... failed runner=Mk5hMxa5 status=couldn't execute POST against https://gitlab.mydomain.com/api/v4/runners: Post https://gitlab.mydomain.com/api/v4/runners: x509: certificate is valid for ingress.local, not gitlab.mydomain.com
PANIC: Failed to register this runner. Perhaps you are having network problems

使用kubectl describe certificate gitlab-gitlab-tls -n gitlab描述证书：

Name:         gitlab-gitlab-tls
Namespace:    gitlab
Labels:       app=unicorn
              chart=unicorn-2.4.6
              heritage=Tiller
              io.cattle.field/appId=gitlab
              release=gitlab
Annotations:  <none>
API Version:  certmanager.k8s.io/v1alpha1
Kind:         Certificate
Metadata:
  Creation Timestamp:  2019-11-13T13:49:10Z
  Generation:          3
  Owner References:
    API Version:           extensions/v1beta1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  gitlab-unicorn
    UID:                   5640645f-550b-4073-bdf0-df8b089b0c94
  Resource Version:        6824
  Self Link:               /apis/certmanager.k8s.io/v1alpha1/namespaces/gitlab/certificates/gitlab-gitlab-tls
  UID:                     30ac32bd-c7f3-4f9b-9e3b-966b6090e1a9
Spec:
  Acme:
    Config:
      Domains:
        gitlab.mydomain.com
      http01:
        Ingress Class:  gitlab-nginx
  Dns Names:
    gitlab.mydomain.com
  Issuer Ref:
    Kind:       Issuer
    Name:       gitlab-issuer
  Secret Name:  gitlab-gitlab-tls
Status:
  Conditions:
    Last Transition Time:  2019-11-13T13:49:10Z
    Message:               Certificate issuance in progress. Temporary certificate issued.
    Reason:                TemporaryCertificate
    Status:                False
    Type:                  Ready
Events:                    <none>

使用kubectl describe issuer gitlab-issuer -n gitlab对发行者的描述

Name:         gitlab-issuer
Namespace:    gitlab
Labels:       app=certmanager-issuer
              chart=certmanager-issuer-0.1.0
              heritage=Tiller
              release=gitlab
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"certmanager.k8s.io/v1alpha1","kind":"Issuer","metadata":{"annotations":{},"creationTimestamp":"2019-11-13T13:49:10Z","gener...
API Version:  certmanager.k8s.io/v1alpha1
Kind:         Issuer
Metadata:
  Creation Timestamp:  2019-11-13T13:49:10Z
  Generation:          4
  Resource Version:    24537
  Self Link:           /apis/certmanager.k8s.io/v1alpha1/namespaces/gitlab/issuers/gitlab-issuer
  UID:                 b9971d7a-5220-47ca-a7f9-607aa3f9be4f
Spec:
  Acme:
    Email:  mh@mydomain.com
    http01:
    Private Key Secret Ref:
      Name:  gitlab-acme-key
    Server:  https://acme-v02.api.letsencrypt.org/directory
Status:
  Acme:
    Last Registered Email:  mh@mydomain.com
    Uri:                    https://acme-v02.api.letsencrypt.org/acme/acct/71695690
  Conditions:
    Last Transition Time:  2019-11-13T13:49:12Z
    Message:               The ACME account was registered with the ACME server
    Reason:                ACMEAccountRegistered
    Status:                True
    Type:                  Ready
Events:                    <none>

使用kubectl describe challenges.certmanager.k8s.io -n gitlab gitlab-gitlab-tls-3386074437-0描述挑战

Name:         gitlab-gitlab-tls-3386074437-0
Namespace:    gitlab
Labels:       acme.cert-manager.io/order-name=gitlab-gitlab-tls-3386074437
Annotations:  <none>
API Version:  certmanager.k8s.io/v1alpha1
Kind:         Challenge
Metadata:
  Creation Timestamp:  2019-11-13T13:49:15Z
  Finalizers:
    finalizer.acme.cert-manager.io
  Generation:  4
  Owner References:
    API Version:           certmanager.k8s.io/v1alpha1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Order
    Name:                  gitlab-gitlab-tls-3386074437
    UID:                   1f01771e-2e38-491f-9b2d-ab5f4fda60e2
  Resource Version:        6915
  Self Link:               /apis/certmanager.k8s.io/v1alpha1/namespaces/gitlab/challenges/gitlab-gitlab-tls-3386074437-0
  UID:                     4c115a6f-a76f-4859-a5db-6acd9c039d71
Spec:
  Authz URL:  https://acme-v02.api.letsencrypt.org/acme/authz-v3/1220588820
  Config:
    http01:
      Ingress Class:  gitlab-nginx
  Dns Name:           gitlab.mydomain.com
  Issuer Ref:
    Kind:    Issuer
    Name:    gitlab-issuer
  Key:       lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8.lPWns02SmS3zXwFzHdma_RyhwwlzWLRDkdlugFXDlZY
  Token:     lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8
  Type:      http-01
  URL:       https://acme-v02.api.letsencrypt.org/acme/chall-v3/1220588820/AwsnPw
  Wildcard:  false
Status:
  Presented:   true
  Processing:  true
  Reason:      Waiting for http-01 challenge propagation: wrong status code '404', expected '200'
  State:       pending
Events:        <none>

在cert-manager pod中找到的日志：

I1113 14:20:21.857235       1 pod.go:58] cert-manager/controller/challenges/http01/selfCheck/http01/ensurePod "level"=0 "msg"="found one existing HTTP01 solver pod" "dnsName"="gitlab.mydomain.com" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-ttkmj" "related_resource_namespace"="gitlab" "resource_kind"="Challenge" "resource_name"="gitlab-gitlab-tls-3386074437-0" "resource_namespace"="gitlab" "type"="http-01" 
I1113 14:20:21.857458       1 service.go:43] cert-manager/controller/challenges/http01/selfCheck/http01/ensureService "level"=0 "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="gitlab.mydomain.com" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-sdlw7" "related_resource_namespace"="gitlab" "resource_kind"="Challenge" "resource_name"="gitlab-gitlab-tls-3386074437-0" "resource_namespace"="gitlab" "type"="http-01" 
I1113 14:20:21.857592       1 ingress.go:91] cert-manager/controller/challenges/http01/selfCheck/http01/ensureIngress "level"=0 "msg"="found one existing HTTP01 solver ingress" "dnsName"="gitlab.mydomain.com" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-7jzwk" "related_resource_namespace"="gitlab" "resource_kind"="Challenge" "resource_name"="gitlab-gitlab-tls-3386074437-0" "resource_namespace"="gitlab" "type"="http-01" 
E1113 14:20:21.864785       1 sync.go:183] cert-manager/controller/challenges "msg"="propagation check failed" "error"="wrong status code '404', expected '200'" "dnsName"="gitlab.mydomain.com" "resource_kind"="Challenge" "resource_name"="gitlab-gitlab-tls-3386074437-0" "resource_namespace"="gitlab" "type"="http-01" 

• DNS gitlab.mydomain.com被设置为指向NGINX正在运行的LoadBalancer的IP。
• 如果我在浏览器中转到https://gitlab.mydomain.com：
  • 浏览器说连接是不安全的
  • 结果是“默认后端- 404”。

​

编辑

基于kubectl describe svc gitlab-nginx-ingress-controller -n gitlab的入口控制器的描述

Name:              gitlab-nginx-ingress-controller
Namespace:         gitlab
Labels:            app=nginx-ingress
                   chart=nginx-ingress-0.30.0-1
                   component=controller
                   heritage=Tiller
                   io.cattle.field/appId=gitlab
                   release=gitlab
Annotations:       field.cattle.io/ipAddresses: null
                   field.cattle.io/targetDnsRecordIds: null
                   field.cattle.io/targetWorkloadIds: null
Selector:          <none>
Type:              ExternalName
IP:
External Name:     gitlab.mydomain.com
Port:              http  80/TCP
TargetPort:        http/TCP
NodePort:          http  31487/TCP
Endpoints:         10.42.0.7:80,10.42.1.9:80,10.42.2.12:80
Port:              https  443/TCP
TargetPort:        https/TCP
NodePort:          https  31560/TCP
Endpoints:         10.42.0.7:443,10.42.1.9:443,10.42.2.12:443
Port:              gitlab-shell  22/TCP
TargetPort:        gitlab-shell/TCP
NodePort:          gitlab-shell  30539/TCP
Endpoints:         10.42.0.7:22,10.42.1.9:22,10.42.2.12:22
Session Affinity:  None
Events:            <none>

运行kubectl get ingress -n gitlab会给我一堆入口：

NAME                        HOSTS                 ADDRESS             PORTS     AGE
cm-acme-http-solver-5rjg4   minio.mydomain.com      gitlab.mydomain.com   80        4d23h
cm-acme-http-solver-7jzwk   gitlab.mydomain.com     gitlab.mydomain.com   80        4d23h
cm-acme-http-solver-tzs25   registry.mydomain.com   gitlab.mydomain.com   80        4d23h
gitlab-minio                minio.mydomain.com      gitlab.mydomain.com   80, 443   4d23h
gitlab-registry             registry.mydomain.com   gitlab.mydomain.com   80, 443   4d23h
gitlab-unicorn              gitlab.mydomain.com     gitlab.mydomain.com   80, 443   4d23h

使用gitlab-unicorn对kubectl describe ingress gitlab-unicron -n gitlab的描述

Name:             gitlab-unicorn
Namespace:        gitlab
Address:          gitlab.mydomain.com
Default backend:  default-http-backend:80 (<none>)
TLS:
  gitlab-gitlab-tls terminates gitlab.mydomain.com
Rules:
  Host               Path  Backends
  ----               ----  --------
  gitlab.mydomain.com
                     /                gitlab-unicorn:8181 (10.42.0.9:8181,10.42.1.8:8181)
                     /admin/sidekiq   gitlab-unicorn:8080 (10.42.0.9:8080,10.42.1.8:8080)
Annotations:
  certmanager.k8s.io/issuer:                          gitlab-issuer
  field.cattle.io/publicEndpoints:                    [{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"gitlab:gitlab-unicorn","ingressName":"gitlab:gitlab-unicorn","hostname":"gitlab.mydomain.com","path":"/","allNodes":false},{"addresses":[""],"port":443,"protocol":"HTTPS","serviceName":"gitlab:gitlab-unicorn","ingressName":"gitlab:gitlab-unicorn","hostname":"gitlab.mydomain.com","path":"/admin/sidekiq","allNodes":false}]
  kubernetes.io/ingress.class:                        gitlab-nginx
  kubernetes.io/ingress.provider:                     nginx
  nginx.ingress.kubernetes.io/proxy-body-size:        512m
  nginx.ingress.kubernetes.io/proxy-connect-timeout:  15
  nginx.ingress.kubernetes.io/proxy-read-timeout:     600
Events:                                               <none>

用cm-acme-http-solver-7jzwk描述kubectl describe ingress cm-acme-http-solver-7jzwk -n gitlab

Name:             cm-acme-http-solver-7jzwk
Namespace:        gitlab
Address:          gitlab.mydomain.com
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host               Path  Backends
  ----               ----  --------
  gitlab.mydomain.com
                     /.well-known/acme-challenge/lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8   cm-acme-http-solver-sdlw7:8089 (10.42.2.19:8089)
Annotations:
  field.cattle.io/publicEndpoints:                     [{"addresses":[""],"port":80,"protocol":"HTTP","serviceName":"gitlab:cm-acme-http-solver-sdlw7","ingressName":"gitlab:cm-acme-http-solver-7jzwk","hostname":"gitlab.mydomain.com","path":"/.well-known/acme-challenge/lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8","allNodes":false}]
  kubernetes.io/ingress.class:                         gitlab-nginx
  nginx.ingress.kubernetes.io/whitelist-source-range:  0.0.0.0/0,::/0
Events:                                                <none>

在我的LoadBalancer和集群的每个节点上打开端口(我知道我应该关闭一些端口，但我将首先设法使我的gitlab设置工作)：

80/tcp                     ALLOW       Anywhere
443/tcp                    ALLOW       Anywhere
22/tcp                     ALLOW       Anywhere
2376/tcp                   ALLOW       Anywhere
2379/tcp                   ALLOW       Anywhere
2380/tcp                   ALLOW       Anywhere
6443/tcp                   ALLOW       Anywhere
6783/tcp                   ALLOW       Anywhere
6783:6784/udp              ALLOW       Anywhere
8472/udp                   ALLOW       Anywhere
4789/udp                   ALLOW       Anywhere
9099/tcp                   ALLOW       Anywhere
10250/tcp                  ALLOW       Anywhere
10254/tcp                  ALLOW       Anywhere
30000:32767/tcp            ALLOW       Anywhere
30000:32767/udp            ALLOW       Anywhere
80/tcp (v6)                ALLOW       Anywhere (v6)
443/tcp (v6)               ALLOW       Anywhere (v6)
22/tcp (v6)                ALLOW       Anywhere (v6)
2376/tcp (v6)              ALLOW       Anywhere (v6)
2379/tcp (v6)              ALLOW       Anywhere (v6)
2380/tcp (v6)              ALLOW       Anywhere (v6)
6443/tcp (v6)              ALLOW       Anywhere (v6)
6783/tcp (v6)              ALLOW       Anywhere (v6)
6783:6784/udp (v6)         ALLOW       Anywhere (v6)
8472/udp (v6)              ALLOW       Anywhere (v6)
4789/udp (v6)              ALLOW       Anywhere (v6)
9099/tcp (v6)              ALLOW       Anywhere (v6)
10250/tcp (v6)             ALLOW       Anywhere (v6)
10254/tcp (v6)             ALLOW       Anywhere (v6)
30000:32767/tcp (v6)       ALLOW       Anywhere (v6)
30000:32767/udp (v6)       ALLOW       Anywhere (v6)

kubectl get pods -n gitlab

cm-acme-http-solver-4d8s5                               1/1     Running            0          5d
cm-acme-http-solver-ttkmj                               1/1     Running            0          5d
cm-acme-http-solver-ws7kv                               1/1     Running            0          5d
gitlab-certmanager-57bc6fb4fd-6rfds                     1/1     Running            0          5d
gitlab-gitaly-0                                         1/1     Running            0          5d
gitlab-gitlab-exporter-57b99467d4-knbgk                 1/1     Running            0          5d
gitlab-gitlab-runner-64b74bcd59-mxwvm                   0/1     CrashLoopBackOff   10         55m
gitlab-gitlab-shell-cff8b68f7-zng2c                     1/1     Running            0          5d
gitlab-gitlab-shell-cff8b68f7-zqvfr                     1/1     Running            0          5d
gitlab-issuer.1-lqs7c                                   0/1     Completed          0          5d
gitlab-migrations.1-c4njn                               0/1     Completed          0          5d
gitlab-minio-75567fcbb6-jjxhw                           1/1     Running            6          5d
gitlab-minio-create-buckets.1-6zljh                     0/1     Completed          0          5d
gitlab-nginx-ingress-controller-698fbc4c64-4wt97        1/1     Running            0          5d
gitlab-nginx-ingress-controller-698fbc4c64-5kv2h        1/1     Running            0          5d
gitlab-nginx-ingress-controller-698fbc4c64-jxljq        1/1     Running            0          5d
gitlab-nginx-ingress-default-backend-6cd54c5f86-2jrkd   1/1     Running            0          5d
gitlab-nginx-ingress-default-backend-6cd54c5f86-cxlmx   1/1     Running            0          5d
gitlab-postgresql-66d8d9574b-hbx78                      2/2     Running            0          5d
gitlab-prometheus-server-6fb685b9c7-c8bqj               2/2     Running            0          5d
gitlab-redis-7668c4d476-tcln5                           2/2     Running            0          5d
gitlab-registry-7bb984c765-7ww6j                        1/1     Running            0          5d
gitlab-registry-7bb984c765-t5jjq                        1/1     Running            0          5d
gitlab-sidekiq-all-in-1-8fd95bf7b-hfnjz                 1/1     Running            0          5d
gitlab-task-runner-5cd7bf5bb9-gnv8p                     1/1     Running            0          5d
gitlab-unicorn-864bd864f5-47zxg                         2/2     Running            0          5d
gitlab-unicorn-864bd864f5-gjms2                         2/2     Running            0          5d

它们是3支-http-解决器：

• 一张给registry.mydomain.com
• 一张给minio.mydomain.com
• 一张给gitlab.mydomain.com

指向gitlab.mydomain.com的日志

I1113 13:49:21.207782       1 solver.go:39] cert-manager/acmesolver "level"=0 "msg"="starting listener"  "expected_domain"="gitlab.mydomain.com" "expected_key"="lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8.lPWns02SmS3zXwFzHdma_RyhwwlzWLRDkdlugFXDlZY" "expected_token"="lSJdy9Os7BmI56EQCkcEl8t36pcR1hWNjri2Vvq0iv8" "listen_port"=8089

kubectl get svc -n gitlab结果

cm-acme-http-solver-48b2j                 NodePort       10.43.58.52     <none>              8089:30090/TCP                            5d23h
cm-acme-http-solver-h42mk                 NodePort       10.43.23.141    <none>              8089:30415/TCP                            5d23h
cm-acme-http-solver-sdlw7                 NodePort       10.43.86.27     <none>              8089:32309/TCP                            5d23h
gitlab-gitaly                             ClusterIP      None            <none>              8075/TCP,9236/TCP                         5d23h
gitlab-gitlab-exporter                    ClusterIP      10.43.187.247   <none>              9168/TCP                                  5d23h
gitlab-gitlab-shell                       ClusterIP      10.43.246.124   <none>              22/TCP                                    5d23h
gitlab-minio-svc                          ClusterIP      10.43.117.249   <none>              9000/TCP                                  5d23h
gitlab-nginx-ingress-controller           ExternalName   <none>          gitlab.mydomain.com   80:31487/TCP,443:31560/TCP,22:30539/TCP   5d23h
gitlab-nginx-ingress-controller-metrics   ClusterIP      10.43.152.252   <none>              9913/TCP                                  5d23h
gitlab-nginx-ingress-controller-stats     ClusterIP      10.43.173.191   <none>              18080/TCP                                 5d23h
gitlab-nginx-ingress-default-backend      ClusterIP      10.43.116.121   <none>              80/TCP                                    5d23h
gitlab-postgresql                         ClusterIP      10.43.97.139    <none>              5432/TCP                                  5d23h
gitlab-prometheus-server                  ClusterIP      10.43.67.220    <none>              80/TCP                                    5d23h
gitlab-redis                              ClusterIP      10.43.36.138    <none>              6379/TCP,9121/TCP                         5d23h
gitlab-registry                           ClusterIP      10.43.54.244    <none>              5000/TCP                                  5d23h
gitlab-unicorn                            ClusterIP      10.43.76.61     <none>              8080/TCP,8181/TCP                         5d23h

pod gitlab-nginx-ingress-controller-698fbc4c64-jxljq的日志(其他nginx-ingress控制器提供相同的日志)：https://textuploader.com/1o9we

对我的配置有什么可能出错的提示吗？

可以自由地要求更多关于我的设置的信息。

非常感谢。

Answer 1

好吧，问题是，Gitlab需要一个有效的SSL证书来解决这个问题。而根据以下的输出，你似乎没有：

E1113 14:20:21.864785       1 sync.go:183] cert-manager/controller/challenges "msg"="propagation check failed" "error"="wrong status code '404', expected '200'" "dnsName"="gitlab.mydomain.com" "resource_kind"="Challenge" "resource_name"="gitlab-gitlab-tls-3386074437-0" "resource_namespace"="gitlab" "type"="http-01" 

Status:
  Presented:   true
  Processing:  true
  Reason:      Waiting for http-01 challenge propagation: wrong status code '404', expected '200'
  State:       pending

http-01的挑战是它将尝试对您的域执行web请求，并且它应该返回一个200 HTTP响应。当您自己说https://gitlab.mydomain.com给您一个404响应时(因此它将无法发出有效的证书)。要进一步诊断这一点，请检查负责域的入口的输出，然后沿着“链”跟踪，直到确定404的响应位置为止。

Answer 2

http01挑战依赖于端口80 (http)来公开以回答挑战。选项controller.service.enableHttp配置http，默认情况下启用(参见这里 )。但是，即使您没有触及此配置，也可能有一个上游组件(即防火墙)阻塞端口80上的通信量。

您能检查一下您的入口Service是否正在监听端口80，是否可以从互联网上联系到？您可以尝试通过浏览器访问端口80上的公共IP，以检查是否从入口控制器(或后端)获得响应。