Nginx- TCP入口点不起作用

Question

我使用Nginx作为Kubernetes侵入控制器。在遵循这简单示例之后，我能够设置这个示例

现在，我正尝试使用以下配置设置logstash的TCP入口点

Logstash：

apiVersion: v1
kind: Secret
metadata:
  name: logstash-secret
  namespace: kube-logging
type: Opaque
data:
  tls.crt: "<base64 encoded>" #For logstash.test.domain.com
  tls.key: "<base64 encoded>" #For logstash.test.domain.com

---

apiVersion: v1
kind: ConfigMap
metadata:
  name: logstash-config
  namespace: kube-logging
  labels:
    app: logstash
data:
  syslog.conf: |-
    input {
      tcp {
          port => 5050
          type => syslog
      }
    }

    filter {
        grok {
          match => {"message" => "%{SYSLOGLINE}"}
        }
    }

    output {
      elasticsearch {
        hosts => ["http://elasticsearch:9200"] #elasticsearch running in same namespace (kube-logging)
        index => "syslog-%{+YYYY.MM.dd}"
      }
      stdout { codec => rubydebug }
    }

---

kind: Deployment
apiVersion: apps/v1
metadata:
  name: logstash
  namespace: kube-logging
  labels:
    app: logstash
spec:
  replicas: 1
  selector:
    matchLabels:
      app: logstash
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: logstash
    spec:
      #serviceAccountName: logstash
      containers:
      - name: logstash
        image: docker.elastic.co/logstash/logstash:7.2.1
        imagePullPolicy: Always
        env:
        - name: ELASTICSEARCH_HOST
          value: elasticsearch
        - name: ELASTICSEARCH_PORT
          value: "9200"
        - name: ELASTICSEARCH_USERNAME
          value: elastic
        - name: ELASTICSEARCH_PASSWORD
          value: changeme
        - name: ELASTIC_CLOUD_ID
          value:
        - name: ELASTIC_CLOUD_AUTH
          value:
        ports:
        - name: logstash
          containerPort: 5050
          protocol: TCP
        securityContext:
          runAsUser: 0
        volumeMounts:
        - name: config
          mountPath: /usr/share/logstash/pipeline/syslog.conf
          readOnly: true
          subPath: syslog.conf
      volumes:
      - name: config
        configMap:
          defaultMode: 0600
          name: logstash-config

---

kind: Service
apiVersion: v1
metadata:
  name: logstash
  namespace: kube-logging
  labels:
    app: logstash
spec:
  ports:
    - name: tcp-port
      protocol: TCP
      port: 5050
      targetPort: 5050
  selector:
    app: logstash

Nginx-侵入

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress
  namespace: kube-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
    spec:
      serviceAccountName: nginx-ingress
      containers:
      - image: nginx/nginx-ingress:1.5.7
        imagePullPolicy: Always
        name: nginx-ingress
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
        - name: tcp5050
          containerPort: 5050
        securityContext:
          allowPrivilegeEscalation: true
          runAsUser: 101 #nginx
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        args:
          - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
          - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
          - -v=3 # Enables extensive logging. Useful for troubleshooting.
         #- -report-ingress-status
         #- -external-service=nginx-ingress
         #- -enable-leader-election
         #- -enable-prometheus-metrics
         #- -enable-custom-resources

LoadBalancer：

apiVersion: v1
kind: Service
metadata:
  name: nginx-ingress-external
  namespace: kube-ingress
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
spec:
  type: LoadBalancer
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80
  - name: https
    protocol: TCP
    port: 443
    targetPort: 443
  - name: tcp5050
    protocol: TCP
    port: 5050
    targetPort: 5050
  selector:
    app: nginx-ingress

Ingress：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: logstash-ingress
  namespace: kube-logging
spec:
  tls:
  - hosts:
    - logstash.test.domain.com
    secretName: logstash-secret #This has self-signed cert for logstash.test.domain.com
  rules:
  - host: logstash.test.domain.com
    http:
      paths:
      - path: /
        backend:
          serviceName: logstash
          servicePort: 5050

使用此配置，它将显示如下：

NAME               HOSTS                         ADDRESS   PORTS     AGE
logstash-ingress   logstash.test.domain.com                 80, 443   79m

为什么这里没有列出5050港口？

只想通过公共端点公开logstash服务。当我在集群中使用openssl s_client -connect logstash.kube-logging.svc.cluster.local:5050时，我得到

$ openssl s_client -connect logstash.kube-logging.svc.cluster.local:5050
CONNECTED(00000005)

但是在集群openssl s_client -connect logstash.test.domain.com:5050之外，我得到了

$ openssl s_client -connect logstash.test.domain.com:5050
connect: Connection refused
connect:errno=61

和

$ openssl s_client -cert logstash_test_domain_com.crt -key logstash_test_domain_com.key -servername logstash.test.domain.com:5050
connect: Connection refused
connect:errno=61

我该怎么做才能让这件事奏效？

Answer 1

你好像有点困惑。所以，让我们从订购您的服务和入口开始。

首先，kubernetes有3种类型的服务。ClusterIP允许您在k8s内部公开部署。Nodeport与ClusterIP相同，但也通过每个节点、外部IP和一个在30K-32K范围内的端口公开您的部署。最后，还有LoadBalancer服务，它与ClusterIP相同，但也在云提供商LoadBalancer分配的特定外部IP地址中公开应用程序。

您创建的NodePort服务将使logstash可以通过30K至32K随机端口中的每个节点外部IP访问；找到运行kubectl get services | grep nginx-ingress的端口并检查最后一列。要获取节点的外部ip地址，请运行kubectl get node -o wide。您创建的LoadBalancer服务将通过端口5050中的外部IP地址访问logstash。要查找IP运行kubectl get services | grep nginx-ingress-external。最后，您还创建了一个入口资源来访问logstash。为此，您已经定义了一个主机，它可以在端口443中访问TLS，并将入站通信量重定向到端口5050中的logstash服务类型ClusterIP。在这里，你有三种方法来达到对数储存。考虑到LoadBalancer是一个特定的端口，我会选择它。