如何在AWS EKS中将kube-system节点安排到fargate？

Question

我通过terraform将EKS集群部署到AWS。有两个fargate配置文件，一个用于kube-system，另一个是default。集群创建完成后，kube-system下的所有pods都处于挂起状态。错误是：

$ kubectl get pods -A
NAMESPACE     NAME                       READY   STATUS    RESTARTS   AGE
kube-system   coredns-6db676b456-694w8   0/1     Pending   0          3m43s
kube-system   coredns-6db676b456-tddtd   0/1     Pending   0          3m43s
kube-system   coredns-b8f47f545-7wzm8    0/1     Pending   0          78m

$ kubectl describe --namespace kube-system pod coredns-6db676b456-694w8
Warning  FailedScheduling  21s (x3 over 92s)  default-scheduler  no nodes available to schedule pods

法吉特似乎没有被安排到这些吊舱里。我做错什么了？下面是完整的terrform配置：

resource "aws_eks_cluster" "elk" {
  name     = "elk"
  role_arn = aws_iam_role.elk.arn
  version  = 1.20

  vpc_config {
    subnet_ids = [module.vpc.private_subnets[0], module.vpc.private_subnets[1]]
  }

  # Ensure that IAM Role permissions are created before and deleted after EKS Cluster handling.
  # Otherwise, EKS will not be able to properly delete EKS managed EC2 infrastructure such as Security Groups.
  depends_on = [
    aws_iam_role_policy_attachment.elk-AmazonEKSClusterPolicy,
    aws_iam_role_policy_attachment.elk-AmazonEKSVPCResourceController,
  ]
}

output "endpoint" {
  value = aws_eks_cluster.elk.endpoint
}

output "kubeconfig-certificate-authority-data" {
  value = aws_eks_cluster.elk.certificate_authority[0].data
}

# Fargate

resource "aws_eks_fargate_profile" "elk" {
  cluster_name           = aws_eks_cluster.elk.name
  fargate_profile_name   = "elk_profile"
  pod_execution_role_arn = aws_iam_role.fargate_profile.arn
  subnet_ids             = [module.vpc.private_subnets[0], module.vpc.private_subnets[1]]

  selector {
    namespace = "default"
  }
}

resource "aws_eks_fargate_profile" "kube_system" {
  cluster_name           = aws_eks_cluster.elk.name
  fargate_profile_name   = "kube_system_profile"
  pod_execution_role_arn = aws_iam_role.fargate_profile.arn
  subnet_ids             = [module.vpc.private_subnets[0], module.vpc.private_subnets[1]]

  selector {
    namespace = "kube-system"
  }
}

# IAM Role

resource "aws_iam_role" "elk" {
  name = "eks-cluster-elk"

  assume_role_policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "eks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
POLICY
}

resource "aws_iam_role_policy_attachment" "elk-AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.elk.name
}

# Optionally, enable Security Groups for Pods
# Reference: https://docs.aws.amazon.com/eks/latest/userguide/security-groups-for-pods.html
resource "aws_iam_role_policy_attachment" "elk-AmazonEKSVPCResourceController" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
  role       = aws_iam_role.elk.name
}

# IAM role for service account

data "tls_certificate" "elk" {
  url = aws_eks_cluster.elk.identity[0].oidc[0].issuer
}

resource "aws_iam_openid_connect_provider" "elk" {
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = [data.tls_certificate.elk.certificates[0].sha1_fingerprint]
  url             = aws_eks_cluster.elk.identity[0].oidc[0].issuer
}

data "aws_iam_policy_document" "elk_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]
    effect  = "Allow"

    condition {
      test     = "StringEquals"
      variable = "${replace(aws_iam_openid_connect_provider.elk.url, "https://", "")}:sub"
      values   = ["system:serviceaccount:kube-system:aws-node"]
    }

    principals {
      identifiers = [aws_iam_openid_connect_provider.elk.arn]
      type        = "Federated"
    }
  }
}

# resource "aws_iam_role" "elk" {
#   assume_role_policy = data.aws_iam_policy_document.elk_assume_role_policy.json
#   name               = "elk"
# }

# IAM role for fargate profile

resource "aws_iam_role" "fargate_profile" {
  name = "eks-fargate-profile"

  assume_role_policy = jsonencode({
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "eks-fargate-pods.amazonaws.com"
      }
    }]
    Version = "2012-10-17"
  })
}

resource "aws_iam_role_policy_attachment" "AmazonEKSFargatePodExecutionRolePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSFargatePodExecutionRolePolicy"
  role       = aws_iam_role.fargate_profile.name
}

Answer 1

您可能需要为CoreDNS部署打补丁。默认情况下，它被配置为仅在工作节点上运行，而不是在Fargate上运行。请参阅此doc page中的“(可选)更新CoreDNS”部分