无法在Kubernetes中提供Traefik NestJS应用程序(错误的网关)

Question

我正在尝试使用NestJS为应用程序提供服务，但我无法做到这一点。

我已经将Traefik IngressRoutes配置为同时服务于Traefik Dashboard和ArgoCD (以及更多的测试应用程序)，但是我已经尝试部署这个新应用程序将近2天了，但没有成功。

错误如下：

Bad Gateway

这是Traefik根据请求输出的日志：

[traefik-c88c9f869-b8cm8] 10.0.1.122 - - [11/Dec/2020:03:13:20 +0000] "GET /graphql HTTP/2.0" 502 11 "-" "-" 764 "develop-business-app-64fa6977f85a45bb4625@kubernetescrd" "http://10.0.3.86:8080" 1ms

我不知道是否需要在我的应用程序中进行自定义配置才能使用HTTP/2.0或处理Traefik SSL (因为入口点是websecure)。我一遍又一遍地阅读文档，但总是得到相同的错误(我已经尝试完全删除并重新安装Traefik )

此外，如果我运行kubectl port-forward，我可以像预期的那样使用应用程序。

以下是我的配置文件：

这是我的Traefik部署：

---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: traefik
  labels:
    app.kubernetes.io/name: traefik-proxy
    app.kubernetes.io/version: 1.0.0
    app.kubernetes.io/component: infrastructure
    app.kubernetes.io/part-of: traefik
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: traefik-proxy
  template:
    metadata:
      labels:
        app.kubernetes.io/name: traefik-proxy
        app.kubernetes.io/version: 1.0.0
        app.kubernetes.io/component: infrastructure
        app.kubernetes.io/part-of: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      volumes:
        - name: acme-certificates
          emptyDir: {}
      containers:
        - name: traefik
          image: traefik:v2.3
          args:
            - --accesslog
            - --providers.kubernetescrd
            - --ping
            - --api.dashboard
            - --entrypoints.traefik.address=:8080
            - --entrypoints.web.address=:80
            - --entrypoints.websecure.address=:443
            - --entrypoints.web.http.redirections.entrypoint.to=websecure
            - --entrypoints.websecure.http.tls.certResolver=letsencrypt
            - --certificatesresolvers.letsencrypt.acme.email=accounts+letsencrypt@getbud.co
            - --certificatesresolvers.letsencrypt.acme.storage=/etc/acme/letsencrypt.json
            - --certificatesResolvers.letsencrypt.acme.dnsChallenge.provider=route53
            - --certificatesResolvers.letsencrypt.acme.dnsChallenge.delayBeforeCheck=0
          volumeMounts:
            - name: acme-certificates
              mountPath: /etc/acme
          ports:
            - containerPort: 8080
              name: admin
              protocol: TCP
            - containerPort: 80
              name: web
              protocol: TCP
            - containerPort: 443
              name: websecure
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /ping
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 2
          readinessProbe:
            failureThreshold: 1
            httpGet:
              path: /ping
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 2

这是我的应用程序部署：

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: business-app
  labels:
    app.kubernetes.io/name: business-app
    app.kubernetes.io/version: 1.0.0
    app.kubernetes.io/component: business
    app.kubernetes.io/part-of: application-layer
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: business-app
  template:
    metadata:
      labels:
        app.kubernetes.io/name: business-app
        app.kubernetes.io/version: 1.0.0
        app.kubernetes.io/component: business
        app.kubernetes.io/part-of: application-layer
    spec:
      containers:
        - name: business-app
          image: 904333181156.dkr.ecr.sa-east-1.amazonaws.com/business:$ECR_TAG <- this is updated with the latest tag using envsubst
          ports:
            - containerPort: 8080
              name: web
              protocol: TCP

这是我的应用程序服务：

---
kind: Service
apiVersion: v1
metadata:
  name: business-app
spec:
  selector:
    app.kubernetes.io/name: business-app
  ports:
    - name: web
      port: 80
      targetPort: 8080

这是我的IngressRoute：

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: business-app
  labels:
    app.kubernetes.io/name: business-app
    app.kubernetes.io/version: 1.0.0
    app.kubernetes.io/component: business
    app.kubernetes.io/part-of: application-layer
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`api.develop.getbud.co`)
      kind: Rule
      services:
        - name: business-app
          port: 80
  tls:
    certResolver: letsencrypt
    options: {}

有人能给我一个提示吗?我哪里做错了？

这只是一次更新，我已经将traefik的loglevel更改为debug，以下是它在请求时记录的内容：

[traefik-55888dfd67-r8b2c] time="2020-12-11T04:54:31Z" level=debug msg="Error while Peeking first byte: read tcp 10.0.3.86:80->10.0.1.122:44996: read: connection reset by peer"
[traefik-55888dfd67-r8b2c] time="2020-12-11T04:54:31Z" level=debug msg="Error while Peeking first byte: read tcp 10.0.3.86:8080->10.0.3.100:6380: read: connection reset by peer"
[traefik-55888dfd67-r8b2c] time="2020-12-11T04:54:32Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/graphql\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,pt-BR;q=0.5\"],\"Cache-Control\":[\"no-cache\"],\"Pragma\":[\"no-cache\"],\"Te\":[\"trailers\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (X11; Linux x86_64; rv:83.0) Gecko/20100101 Firefox/83.0\"],\"X-Forwarded-Host\":[\"api.develop.getbud.co\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"traefik-55888dfd67-r8b2c\"],\"X-Real-Ip\":[\"10.0.1.122\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"api.develop.getbud.co\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"10.0.1.122:27473\",\"RequestURI\":\"/graphql\",\"TLS\":null}"
[traefik-55888dfd67-r8b2c] time="2020-12-11T04:54:32Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/graphql\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,pt-BR;q=0.5\"],\"Cache-Control\":[\"no-cache\"],\"Pragma\":[\"no-cache\"],\"Te\":[\"trailers\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (X11; Linux x86_64; rv:83.0) Gecko/20100101 Firefox/83.0\"],\"X-Forwarded-Host\":[\"api.develop.getbud.co\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"traefik-55888dfd67-r8b2c\"],\"X-Real-Ip\":[\"10.0.1.122\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"api.develop.getbud.co\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"10.0.1.122:27473\",\"RequestURI\":\"/graphql\",\"TLS\":null}" ForwardURL="http://10.0.1.158:8080"
[traefik-55888dfd67-r8b2c] time="2020-12-11T04:54:32Z" level=debug msg="'502 Bad Gateway' caused by: dial tcp 10.0.1.158:8080: connect: connection refused"
[traefik-55888dfd67-r8b2c] time="2020-12-11T04:54:32Z" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/graphql\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,pt-BR;q=0.5\"],\"Cache-Control\":[\"no-cache\"],\"Pragma\":[\"no-cache\"],\"Te\":[\"trailers\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (X11; Linux x86_64; rv:83.0) Gecko/20100101 Firefox/83.0\"],\"X-Forwarded-Host\":[\"api.develop.getbud.co\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"traefik-55888dfd67-r8b2c\"],\"X-Real-Ip\":[\"10.0.1.122\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"api.develop.getbud.co\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"10.0.1.122:27473\",\"RequestURI\":\"/graphql\",\"TLS\":null}"
[traefik-55888dfd67-r8b2c] 10.0.1.122 - - [11/Dec/2020:04:54:32 +0000] "GET /graphql HTTP/2.0" 502 11 "-" "-" 754 "develop-business-app-64fa6977f85a45bb4625@kubernetescrd" "http://10.0.1.158:8080" 2ms
[traefik-55888dfd67-r8b2c] time="2020-12-11T04:54:32Z" level=debug msg="Error while Peeking first byte: read tcp 10.0.3.86:443->10.0.3.75:35314: read: connection reset by peer"
[traefik-55888dfd67-r8b2c] time="2020-12-11T04:54:32Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/favicon.ico\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"image/webp,*/*\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,pt-BR;q=0.5\"],\"Cache-Control\":[\"no-cache\"],\"Pragma\":[\"no-cache\"],\"Referer\":[\"https://api.develop.getbud.co/graphql\"],\"Te\":[\"trailers\"],\"User-Agent\":[\"Mozilla/5.0 (X11; Linux x86_64; rv:83.0) Gecko/20100101 Firefox/83.0\"],\"X-Forwarded-Host\":[\"api.develop.getbud.co\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"traefik-55888dfd67-r8b2c\"],\"X-Real-Ip\":[\"10.0.1.122\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"api.develop.getbud.co\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"10.0.1.122:27473\",\"RequestURI\":\"/favicon.ico\",\"TLS\":null}"
[traefik-55888dfd67-r8b2c] time="2020-12-11T04:54:32Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" ForwardURL="http://10.0.1.158:8080" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/favicon.ico\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"image/webp,*/*\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,pt-BR;q=0.5\"],\"Cache-Control\":[\"no-cache\"],\"Pragma\":[\"no-cache\"],\"Referer\":[\"https://api.develop.getbud.co/graphql\"],\"Te\":[\"trailers\"],\"User-Agent\":[\"Mozilla/5.0 (X11; Linux x86_64; rv:83.0) Gecko/20100101 Firefox/83.0\"],\"X-Forwarded-Host\":[\"api.develop.getbud.co\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"traefik-55888dfd67-r8b2c\"],\"X-Real-Ip\":[\"10.0.1.122\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"api.develop.getbud.co\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"10.0.1.122:27473\",\"RequestURI\":\"/favicon.ico\",\"TLS\":null}"
[traefik-55888dfd67-r8b2c] time="2020-12-11T04:54:32Z" level=debug msg="'502 Bad Gateway' caused by: dial tcp 10.0.1.158:8080: connect: connection refused"
[traefik-55888dfd67-r8b2c] time="2020-12-11T04:54:32Z" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/favicon.ico\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"image/webp,*/*\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,pt-BR;q=0.5\"],\"Cache-Control\":[\"no-cache\"],\"Pragma\":[\"no-cache\"],\"Referer\":[\"https://api.develop.getbud.co/graphql\"],\"Te\":[\"trailers\"],\"User-Agent\":[\"Mozilla/5.0 (X11; Linux x86_64; rv:83.0) Gecko/20100101 Firefox/83.0\"],\"X-Forwarded-Host\":[\"api.develop.getbud.co\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"traefik-55888dfd67-r8b2c\"],\"X-Real-Ip\":[\"10.0.1.122\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"api.develop.getbud.co\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"10.0.1.122:27473\",\"RequestURI\":\"/favicon.ico\",\"TLS\":null}"
[traefik-55888dfd67-r8b2c] 10.0.1.122 - - [11/Dec/2020:04:54:32 +0000] "GET /favicon.ico HTTP/2.0" 502 11 "-" "-" 755 "develop-business-app-64fa6977f85a45bb4625@kubernetescrd" "http://10.0.1.158:8080" 1ms

因此，Traefik似乎收到了来自pod的拒绝连接。我已经在Traefik的容器中打开了一个shell，并尝试直接在Pod IP中运行wget，但实际上我收到了相同的错误(连接被拒绝)。

任何其他工作的pod，只要我运行wget它就能工作。

有什么想法吗？

Answer 1

对于那些(像我一样)正在努力解决这个问题的人，这里有一些问题：

默认情况下，Fastify只侦听127.0.0.1，因此它会自动拒绝任何其他主机连接。要解决这个问题，只需添加0.0.0.0作为app.listen调用的第二个参数，如下所示：

上一次：

  await app.listen(appConfig.port)

已修复：

  await app.listen(appConfig.port, '0.0.0.0')

无论如何，感谢您:)