如何使用Terraform配置EKS ALB

Question

我很难让EKS将IP地址暴露在公共互联网上。我需要自己设置ALB吗?还是作为EKS集群的一部分，您可以免费获得？如果我必须自己做，我需要在terraform模板文件或kubernetes对象yaml中定义它吗？

这是我在Terraform中定义的EKS集群，以及我认为需要的权限。

// eks.tf

resource "aws_iam_role" "eks_cluster_role" {
  name = "${local.env_name}-eks-cluster-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Allow",
        Principal = {
          Service = "eks.amazonaws.com"
        },
        Action = "sts:AssumeRole"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_cluster_role.name
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKSVPCResourceController" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
  role       = aws_iam_role.eks_cluster_role.name
}

resource "aws_kms_key" "eks_key" {
  description             = "EKS KMS Key"
  deletion_window_in_days = 7
  enable_key_rotation     = true

  tags = {
    Environment = local.env_name
    Service     = "EKS"
  }
}

resource "aws_kms_alias" "eks_key_alias" {
  target_key_id = aws_kms_key.eks_key.id
  name          = "alias/eks-kms-key-${local.env_name}"
}

resource "aws_eks_cluster" "eks_cluster" {
  name                      = "${local.env_name}-eks-cluster"
  role_arn                  = aws_iam_role.eks_cluster_role.arn
  enabled_cluster_log_types = ["api", "audit", "authenticator", "controllerManager", "scheduler"]

  vpc_config {
    subnet_ids = [aws_subnet.private_a.id, aws_subnet.private_b.id]
  }

  encryption_config {
    resources = ["secrets"]

    provider {
      key_arn = aws_kms_key.eks_key.arn
    }
  }

  tags = {
    Environment = local.env_name
  }
}

resource "aws_iam_role" "eks_node_group_role" {
  name = "${local.env_name}-eks-node-group"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Allow",
        Principal = {
          Service = "ec2.amazonaws.com"
        },
        Action = "sts:AssumeRole"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "eks-node-group-AmazonEKSWorkerNodePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.eks_node_group_role.name
}

resource "aws_iam_role_policy_attachment" "eks-node-group-AmazonEKS_CNI_Policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.eks_node_group_role.name
}

resource "aws_iam_role_policy_attachment" "eks-node-group-AmazonEC2ContainerRegistryReadOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role       = aws_iam_role.eks_node_group_role.name
}

resource "aws_eks_node_group" "eks_node_group" {
  instance_types  = var.node_group_instance_types
  node_group_name = "${local.env_name}-eks-node-group"
  node_role_arn   = aws_iam_role.eks_node_group_role.arn
  cluster_name    = aws_eks_cluster.eks_cluster.name
  subnet_ids      = [aws_subnet.private_a.id, aws_subnet.private_b.id]

  scaling_config {
    desired_size = 1
    max_size     = 1
    min_size     = 1
  }

  // Ensure that IAM Role permissions are created before and deleted after EKS Node Group handling.
  // Otherwise, EKS will not be able to properly delete EC2 Instances and Elastic Network Interfaces.
  depends_on = [
    aws_iam_role_policy_attachment.eks-node-group-AmazonEC2ContainerRegistryReadOnly,
    aws_iam_role_policy_attachment.eks-node-group-AmazonEKS_CNI_Policy,
    aws_iam_role_policy_attachment.eks-node-group-AmazonEKSWorkerNodePolicy,
  ]

下面是我的kubernetes对象yaml：

# hello-kubernetes.yaml

apiVersion: v1
kind: Service
metadata:
  name: hello-kubernetes
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: hello-kubernetes
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-kubernetes
spec:
  replicas: 3
  selector:
    matchLabels:
      app: hello-kubernetes
  template:
    metadata:
      labels:
        app: hello-kubernetes
    spec:
      containers:
      - name: hello-kubernetes
        image: paulbouwer/hello-kubernetes:1.9
        ports:
        - containerPort: 8080
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: hello-ingress
spec:
  backend:
    serviceName: hello-kubernetes
    servicePort: 80

我已经运行了terraform apply，并且集群已经启动并运行。我已经安装了eksctl和kubectl并运行了kubectl apply -f hello-kubernetes.yaml。pods、服务和入口似乎运行正常。

$ kubectl get pods
NAME                                READY   STATUS             RESTARTS   AGE
hello-kubernetes-6cb7cd595b-25bd9   1/1     Running            0          6h13m
hello-kubernetes-6cb7cd595b-lccdj   1/1     Running            0          6h13m
hello-kubernetes-6cb7cd595b-snwvr   1/1     Running            0          6h13m

$ kubectl get services
NAME               TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
hello-kubernetes   LoadBalancer   172.20.102.37   <pending>     80:32086/TCP   6h15m

$ kubectl get ingresses
NAME            CLASS    HOSTS   ADDRESS   PORTS   AGE
hello-ingress   <none>   *                 80      3h45m

我遗漏了什么?它属于哪个文件？

Answer 1

通常的做法是放置ALB并将流量重定向到EKS集群，使用ALB入口控制器对其进行管理。此入口控制器将充当群集和您的ALB之间的通信，这里是非常简单的AWS文档

EKS w/ALB

如果ALB不适合您的应用程序需要，其他解决方案可以使用带有NLB的NGINX入口控制器，如下面的文章所述

NGINX w/NLB

Answer 2

这也发生在我身上，在所有的设置之后，我无法看到入口地址。调试此问题的最佳方法是检查入口控制器的日志。您可以通过以下方式完成此操作：

通过以下命令获取入口控制器po名称: kubectl Get po -n kube-system Check logs for the po using: kubectl logs -n kube-system这将指出您看不到地址的确切问题。

如果您没有找到任何名为ingress的po运行，那么您必须首先创建入口控制器。

Answer 3

您需要按照installation instructions安装AWS Load Balancer Controller；首先您需要创建IAM角色和权限，这可以使用Terraform完成；然后您需要应用Kubernetes Yaml将控制器安装到您的集群中，这可以通过Helm或Kubectl完成。

您还需要了解创建面向公共或私有的负载均衡器所需的subnet tagging。