nginx-入口tcp服务-连接被拒绝

Question

我目前正在部署一个新的kubernetes集群，我希望使用nginx入口从集群外部公开一个mongodb服务。我知道nginx入口通常用于第7层应用程序，但根据官方文档，它也能够在第4层(TCP/UDP)上工作。

https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/

我的mongodb服务是一个可在端口11717 (内部命名空间)上访问的ClusterIP序列化：

​

kubectl get svc -n internal

mongodb         ClusterIP   10.97.63.154    <none>        11717/TCP                        3d20h

telnet 10.97.63.154 11717
Trying 10.97.63.154...
Connected to 10.97.63.154.

​

我真的尝试了每一个可能的组合来实现这个目标，但没有成功。我使用的是nginx-ingress头盔图(后台类型)。

我的nginx-ingress/templates/controller-daemonset.yaml文件：

​

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ingress-nginx-ingress
  namespace: default
  labels:
    app.kubernetes.io/name: nginx-ingress-nginx-ingress
    helm.sh/chart: nginx-ingress-0.13.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/instance: nginx-ingress
spec:
  selector:
    matchLabels:
      app: nginx-ingress-nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress-nginx-ingress
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "9113"
        prometheus.io/scheme: "http"
    spec:
      serviceAccountName: nginx-ingress-nginx-ingress
      terminationGracePeriodSeconds: 30
      hostNetwork: false
      containers:
      - name: nginx-ingress-nginx-ingress
        image: "nginx/nginx-ingress:2.2.0"
        imagePullPolicy: "IfNotPresent"
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: https
          containerPort: 443
          hostPort: 443
        - name: mongodb
          containerPort: 11717
          hostPort: 11717

        - name: prometheus
          containerPort: 9113
        - name: readiness-port
          containerPort: 8081
        readinessProbe:
          httpGet:
            path: /nginx-ready
            port: readiness-port
          periodSeconds: 1
        securityContext:
          allowPrivilegeEscalation: true
          runAsUser: 101 #nginx
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        resources:
          {}
        args:
          - /nginx-ingress-controller
          - -nginx-plus=false
          - -nginx-reload-timeout=60000
          - -enable-app-protect=false
          - -tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
          - -publish-service=$(POD_NAMESPACE)/ingress-nginx
          - -annotations-prefix=nginx.ingress.kubernetes.io
          - -enable-app-protect-dos=false
          - -nginx-configmaps=$(POD_NAMESPACE)/nginx-ingress-nginx-ingress
          - -default-server-tls-secret=$(POD_NAMESPACE)/nginx-ingress-nginx-ingress-default-server-tls
          - -ingress-class=nginx
          - -health-status=false
          - -health-status-uri=/nginx-health
          - -nginx-debug=false
          - -v=1
          - -nginx-status=true
          - -nginx-status-port=8080
          - -nginx-status-allow-cidrs=127.0.0.1
          - -report-ingress-status
          - -external-service=nginx-ingress-nginx-ingress
          - -enable-leader-election=true
          - -leader-election-lock-name=nginx-ingress-nginx-ingress-leader-election
          - -enable-prometheus-metrics=true
          - -prometheus-metrics-listen-port=9113
          - -prometheus-tls-secret=
          - -enable-custom-resources=true
          - -enable-snippets=false
          - -enable-tls-passthrough=false
          - -enable-preview-policies=false
          - -enable-cert-manager=false
          - -enable-oidc=false
          - -ready-status=true
          - -ready-status-port=8081
          - -enable-latency-metrics=false

​

我的nginx-ingress/templates/controller-service.yaml文件：

​

apiVersion: v1
kind: Service
metadata:
  name: nginx-ingress-nginx-ingress
  namespace: default
  labels:
    app.kubernetes.io/name: nginx-ingress-nginx-ingress
    helm.sh/chart: nginx-ingress-0.13.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/instance: nginx-ingress
spec:
  externalTrafficPolicy: Local
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
  - port: 443
    targetPort: 443
    protocol: TCP
    name: https
  - name: mongodb
    port: 11717
    targetPort: 11717
    protocol: TCP
  selector:
    app:  nginx-ingress-nginx-ingress

​

我的nginx入口/模板/tcp-services.yaml文件：

​

apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: default
data:
  "11717": internal/mongodb:11717

​

​

kubectl get pods
NAME                                READY   STATUS    RESTARTS   AGE
nginx-ingress-nginx-ingress-d5vms   1/1     Running   0          61m
nginx-ingress-nginx-ingress-kcs4p   1/1     Running   0          61m
nginx-ingress-nginx-ingress-mnnn2   1/1     Running   0          61m



kubectl get svc -o wide
NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                      AGE    SELECTOR
kubernetes                    ClusterIP      10.96.0.1       <none>        443/TCP                                      4d1h   <none>
nginx-ingress-nginx-ingress   LoadBalancer   10.99.176.220   <pending>     80:31700/TCP,443:31339/TCP,11717:31048/TCP   61m    app=nginx-ingress-nginx-ingress


telnet 10.99.176.220 80
Trying 10.99.176.220...
Connected to 10.99.176.220.
Escape character is '^]'.


telnet 10.99.176.220 11717
Trying 10.99.176.220...
telnet: Unable to connect to remote host: Connection refused

​

我不明白为什么端口11717的连接被拒绝.

我如何实现这个场景：

mongo.myExternalDomain:11717 -> nginx-ingress服务--> nginx-ingress pod --> mongodb服务--> mongodb pod

提前感谢！任何帮助我都会感激的！

Answer 1

我有个简单的问题。也许这个能帮到你。在我的例子中，它位于tcp-services configmap中：

很快就会。而不是这样：

apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: default
data:
  "11717": internal/mongodb:11717

请改为：

apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: default
data:
  "11717": internal/mongodb:11717:PROXY

详细信息：

namespace/service:8000.

• edit

1. 编辑‘tcp -服务’configmap以添加tcp .service 8000: nginx控制器服务，为step1
2. Check /etc/nginx/nginx.conf中的step1
3. Check/etc/nginx/nginx.conf中的tcp服务添加一个端口(端口:8000-> targetPort:8000)，并确认它包含一个带有正确侦听8000的‘服务器’块；namespace/service:8000:PROXY
4. Check /8000服务指令，
5. 重新编辑‘tcp-
6. ’configmap以添加代理协议解码指令，现在tcp/8000服务的k/v变为8000: nginx /nginx/nginx.conf在nginx控制器荚中，与step3相比没有任何变化，仍然是侦听8000；
7. 编辑了一些入口规则(比如更新主机)，
8. 检查/etc/ nginx /nginx.conf再次出现在nginx控制器荚中，现在tcp/8000服务的侦听指令变成侦听8000 proxy_protocol；这是正确的。