多入口控制器不起作用

Question

我正在不同的名称空间中创建多个入口控制器。最初，它在AWS中创建一个负载均衡器，并将pod地址附加到目标组。几天后，它没有将新的pod更新到目标组。我把入口控制器的日志附在这里了。

E0712 15:02:30.516295       1 leaderelection.go:270] error retrieving resource lock namespace1/ingress-controller-leader-alb: configmaps "ingress-controller-le │
│ ader-alb" is forbidden: User "system:serviceaccount:namespace1:fc-serviceaccount-icalb" cannot get resource "configmaps" in API group "" in the namespace "namespace1"

ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: "fc-ingress"
  annotations:
    kubernetes.io/ingress.class: alb-namespace1
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/subnets:
    alb.ingress.kubernetes.io/certificate-arn: 
    alb.ingress.kubernetes.io/ssl-policy:
    alb.ingress.kubernetes.io/security-groups:
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/healthcheck-port: traffic-port
    alb.ingress.kubernetes.io/healthcheck-path: '/'
    alb.ingress.kubernetes.io/healthcheck-timeout-seconds:  '2'
    alb.ingress.kubernetes.io/healthcheck-interval-seconds: '5'
    alb.ingress.kubernetes.io/success-codes: '200'
    alb.ingress.kubernetes.io/healthy-threshold-count: '5'
    alb.ingress.kubernetes.io/unhealthy-threshold-count: '2'
    alb.ingress.kubernetes.io/load-balancer-attributes: access_logs.s3.enabled=false
    alb.ingress.kubernetes.io/load-balancer-attributes: deletion_protection.enabled=false
    alb.ingress.kubernetes.io/load-balancer-attributes: routing.http2.enabled=true
    alb.ingress.kubernetes.io/target-group-attributes: slow_start.duration_seconds=0
    alb.ingress.kubernetes.io/target-group-attributes: deregistration_delay.timeout_seconds=300
    alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=false
  labels:
    app: fc-label-app-ingress
spec:
  rules:
     - host: "hostname1.com"
       http:
        paths:
        - backend:
            serviceName: service1
            servicePort: 80

     - host: "hostname2.com"
       http:
        paths:
        - backend:
            serviceName: service2
            servicePort: 80

     - host: "hostname3.com"
       http:
         paths:
         - backend:
             serviceName: service3
             servicePort: 80

ingress_controller.yaml

# Application Load Balancer (ALB) Ingress Controller Deployment Manifest.
# This manifest details sensible defaults for deploying an ALB Ingress Controller.
# GitHub: https://github.com/kubernetes-sigs/aws-alb-ingress-controller
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: fc-label-app-icalb
  name: fc-ingress-controller-alb
  namespace: namespace1
  # Namespace the ALB Ingress Controller should run in. Does not impact which
  # namespaces it's able to resolve ingress resource for. For limiting ingress
  # namespace scope, see --watch-namespace.
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fc-label-app-icalb
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: fc-label-app-icalb
    spec:
      containers:
        - args:
            # Limit the namespace where this ALB Ingress Controller deployment will
            # resolve ingress resources. If left commented, all namespaces are used.
            - --watch-namespace=namespace1

            # Setting the ingress-class flag below ensures that only ingress resources with the
            # annotation kubernetes.io/ingress.class: "alb" are respected by the controller. You may
            # choose any class you'd like for this controller to respect.
            - --ingress-class=alb-namespace1

            # Name of your cluster. Used when naming resources created
            # by the ALB Ingress Controller, providing distinction between
            # clusters.
            - --cluster-name=$EKS_CLUSTER_NAME

          # AWS VPC ID this ingress controller will use to create AWS resources.
            # If unspecified, it will be discovered from ec2metadata.
            # - --aws-vpc-id=vpc-xxxxxx

            # AWS region this ingress controller will operate in.
            # If unspecified, it will be discovered from ec2metadata.
            # List of regions: http://docs.aws.amazon.com/general/latest/gr/rande.html#vpc_region
            # - --aws-region=us-west-1

            # Enables logging on all outbound requests sent to the AWS API.
            # If logging is desired, set to true.
            # - ---aws-api-debug
            # Maximum number of times to retry the aws calls.
            # defaults to 10.
            # - --aws-max-retries=10
          env:
            # AWS key id for authenticating with the AWS API.
            # This is only here for examples. It's recommended you instead use
            # a project like kube2iam for granting access.
            #- name: AWS_ACCESS_KEY_ID
            #  value: KEYVALUE

            # AWS key secret for authenticating with the AWS API.
            # This is only here for examples. It's recommended you instead use
            # a project like kube2iam for granting access.
            #- name: AWS_SECRET_ACCESS_KEY
            #  value: SECRETVALUE
          # Repository location of the ALB Ingress Controller.
          image: docker.io/amazon/aws-alb-ingress-controller:v1.1.4
          imagePullPolicy: Always
          name: server
          resources: {}
          terminationMessagePath: /dev/termination-log
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      securityContext: {}
      terminationGracePeriodSeconds: 30
      serviceAccountName: fc-serviceaccount-icalb




---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app: fc-label-app-icalb
  name: fc-clusterrole-icalb
rules:
  - apiGroups:
      - ""
      - extensions
    resources:
      - configmaps
      - endpoints
      - events
      - ingresses
      - ingresses/status
      - services
    verbs:
      - create
      - get
      - list
      - update
      - watch
      - patch
  - apiGroups:
      - ""
      - extensions
    resources:
      - nodes
      - pods
      - secrets
      - services
      - namespaces
    verbs:
      - get
      - list
      - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app: fc-label-app-icalb
  name: fc-clusterrolebinding-icalb
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: fc-clusterrole-icalb
subjects:
  - kind: ServiceAccount
    name: fc-serviceaccount-icalb
    namespace: namespace1
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: fc-label-app-icalb
  name: fc-serviceaccount-icalb
  namespace: namespace1

Answer 1

我在AKS上遇到过类似的问题。我有两个Nginx入侵控制器：

1. external-nginx-ingress
2. internal-nginx-ingress

一次只有一个工作，无论是内部工作还是外部工作。

在为每个election-id指定唯一的之后，问题就得到了解决。

我使用以下舵机图表：

Repository    = "https://kubernetes.github.io/ingress-nginx"
Chart         = "ingress-nginx"
Chart_version = "4.1.3"
K8s Version   = "1.22.4"

部署

kubectl get deploy -n ingress
NAME                                READY   UP-TO-DATE   AVAILABLE
external-nginx-ingress-controller   3/3     3            3           
internal-nginx-ingress-controller   1/1     1            1           

IngressClass

kubectl get ingressclass
NAME             CONTROLLER                      PARAMETERS
external-nginx   k8s.io/ingress-nginx            <none>
internal-nginx   k8s.io/internal-ingress-nginx   <none>

面向外部的部署

apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-nginx-ingress-controller
  namespace: ingress
  annotations:
    meta.helm.sh/release-name: external-nginx-ingress
    meta.helm.sh/release-namespace: ingress
spec:
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: external-nginx-ingress
      app.kubernetes.io/name: ingress-nginx
  template:
    spec:
      containers:
        - name: ingress-nginx-external-controller
          image: >-
            k8s.gcr.io/ingress-nginx/controller:v1.2.1
          args:
            - /nginx-ingress-controller
            - >-
              --publish-service=$(POD_NAMESPACE)/external-nginx-ingress-controller
            - '--election-id=external-ingress-controller-leader'
            - '--controller-class=k8s.io/ingress-nginx'
            - '--ingress-class=external-nginx'
            - '--ingress-class-by-name=true'

内部的部署

apiVersion: apps/v1
kind: Deployment
metadata:
  name: internal-nginx-ingress-controller
  namespace: ingress
  annotations:
    meta.helm.sh/release-name: internal-nginx-ingress
    meta.helm.sh/release-namespace: ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: internal-nginx-ingress
      app.kubernetes.io/name: ingress-nginx
  template:
    spec:
      containers:
        - name: ingress-nginx-internal-controller
          image: >-
            k8s.gcr.io/ingress-nginx/controller:v1.2.1
          args:
            - /nginx-ingress-controller
            - >-
              --publish-service=$(POD_NAMESPACE)/internal-nginx-ingress-controller
            - '--election-id=internal-ingress-controller-leader'
            - '--controller-class=k8s.io/internal-ingress-nginx'
            - '--ingress-class=internal-nginx'
            - '--ingress-class-by-name=true'