Terraform数据源aws_subnet不返回找到的匹配子网

Question

我有一个模块，我试图在其中找到AWS子网，然后使用/返回。它的名字是这样的

module "smurf_subnet_grp" {
   source            = "../../modules/networking/subnet_grp_per_az-test"
   vpc_id            = "${module.networking_uswe2.vpc_id}"
   azs               = "${local.az_list_uswe2}"
   private_subnets   = "${var.private_subnets_uswe2}"
}

模块代码：

variable "azs"              { type = "list" }
variable "private_subnets"  { type = "list" }
variable "vpc_id"           {}

# ========== remove special subnets ==============

locals {
  cnt      = "${length(var.private_subnets) - 3}"
  prv_subs = "${slice(var.private_subnets, 0, local.cnt)}"
}

# ========== get subnet details ==================

data "aws_subnet" "self" {
  count         = "${length(local.prv_subs)}"
  vpc_id        = "${var.vpc_id}"
  cidr_block    = "${local.prv_subs[count.index]}"
}

# ========== get subnets by AZ ===================

locals {
  prv_subs0 = "${matchkeys(data.aws_subnet.self.*.id, data.aws_subnet.self.*.availability_zone, list(var.azs[0]))}"
  prv_subs1 = "${matchkeys(data.aws_subnet.self.*.id, data.aws_subnet.self.*.availability_zone, list(var.azs[1]))}"
  prv_subs2 = "${matchkeys(data.aws_subnet.self.*.id, data.aws_subnet.self.*.availability_zone, list(var.azs[2]))}"
}

# ========== select 1 subnet per AZ ==============

resource "random_shuffle" "prv_sub0" {
    input        = ["${local.prv_subs0}"]
    result_count = 1
}
resource "random_shuffle" "prv_sub1" {
    input        = ["${local.prv_subs1}"]
    result_count = 1
}
resource "random_shuffle" "prv_sub2" {
    input        = ["${local.prv_subs2}"]
    result_count = 1
}

# ========== put selected into 1 list ============

locals {
  prv_sub_az = [
       "${random_shuffle.prv_sub0.result}", 
       "${random_shuffle.prv_sub1.result}", 
       "${random_shuffle.prv_sub2.result}"
  ]
}

output "prv_subnet_grp" {
  value = "${local.prv_sub_az}"
}

这就抛出了：

Error: Error refreshing state: 1 error occurred:
    * module.smurf_subnet_grp.data.aws_subnet.self: 6 errors occurred:
    * module.smurf_subnet_grp.data.aws_subnet.self[5]: data.aws_subnet.self.5: no matching subnet found
    * module.smurf_subnet_grp.data.aws_subnet.self[3]: data.aws_subnet.self.3: no matching subnet found
    * module.smurf_subnet_grp.data.aws_subnet.self[0]: data.aws_subnet.self.0: no matching subnet found
    * module.smurf_subnet_grp.data.aws_subnet.self[1]: data.aws_subnet.self.1: no matching subnet found
    * module.smurf_subnet_grp.data.aws_subnet.self[2]: data.aws_subnet.self.2: no matching subnet found
    * module.smurf_subnet_grp.data.aws_subnet.self[4]: data.aws_subnet.self.4: no matching subnet found

如果我为depends_on数据提供程序介绍一个aws_subnet：

data "aws_subnet" "self" {
  count         = "${length(local.prv_subs)}"
  vpc_id        = "${var.vpc_id}"
  cidr_block    = "${local.prv_subs[count.index]}"
  depends_on    = ["null_resource.module_depends_on"]
}

它会像预期的那样工作，但是每次都会重新创建它。

更新#1

为了解决这个问题，我尝试在Hashicorp讨论论坛上实现一个建议，名为：论仿真。理论上说，我遇到的问题是秩序/依赖问题。

用于实现depends_on的代码如下所示，用于我的subnet_grp_per_az-test模块：

/*
    Add the following line to the resource in this module that depends on the completion of external module components:

    depends_on = ["null_resource.module_depends_on"]

    This will force Terraform to wait until the dependant external resources are created before proceeding with the creation of the
    resource that contains the line above.

    This is a hack until Terraform officially support module depends_on.
*/

variable "module_depends_on" {
  default = [""]
}

resource "null_resource" "module_depends_on" {
  triggers = {
    value = "${length(var.module_depends_on)}"
  }
}

Answer 1

请注意这是我的理解。希望M.Atkins能证实。

为什么Terraform没有用于模块的depends_on

• 关于#18239

作为准备，首先要澄清我们对Terraform 模块可能存在的误解(我也有)。

目标是确定不依赖于TF模块到另一个TF模块。仅仅因为模块A声明出现在模块B之前，就像在下面的根模块 tf文件中那样，并不意味着在模块B中资源的创建将不会发生，直到模块A中的资源全部完成。

如果我们有两个模组A和B，并且它们相互依赖，会发生什么情况？

​

​

根模块

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr
}
#--------------------------------------------------------------------------------
# Create PRIVATE subnets but create EC2 in PUBLIC subnets (cross module reference)
#--------------------------------------------------------------------------------
module "private_subnet_public_ec2" {
  source               = "../private_subnet_public_ec2"
  vpc_id               = aws_vpc.this.id
  private_subnet_cidr_blocks = var.private_subnet_cidr_blocks
  public_subnet_ids    = module.public_subnet_private_ec2.public_subnet_ids
  ami_id               = data.aws_ami.this.id
}
#--------------------------------------------------------------------------------
# Create PUBLIC subnets but create EC2 in PRIVATE subnets (cross module reference)
#--------------------------------------------------------------------------------
module "public_subnet_private_ec2" {
  source              = "../public_subnet_private_ec2"
  vpc_id              = aws_vpc.this.id
  public_subnet_cidr_blocks = var.public_subnet_cidr_blocks
  private_subnet_ids  = module.private_subnet_public_ec2.private_subnet_ids
  ami_id              = data.aws_ami.this.id
}

单元A

在模块B中创建的公共子网中创建私有子网和EC2s。

resource "aws_subnet" "private" {
  count = length(var.private_subnet_cidr_blocks)
  vpc_id     = var.vpc_id
  cidr_block = var.private_subnet_cidr_blocks[count.index]
}
output "private_subnet_ids" {
  value = aws_subnet.private[*].id
}

resource "aws_instance" "public_ec2" {
  count = length(var.public_subnet_ids)
  subnet_id = var.public_subnet_ids[count.index]
  ami           = var.ami_id
  instance_type = "t2.micro"
  tags = {
    Name = "PublicEC2${count.index}}"
  }
  provisioner "local-exec" {
    command = <<EOF
echo "Public EC2 ${count.index} ID is ${self.id}"
EOF
  }
}

单元B

在模块A中创建的私有子网中创建公共子网和EC2s。

resource "aws_subnet" "public" {
  count = length(var.public_subnet_cidr_blocks)
  vpc_id     = var.vpc_id
  cidr_block = var.public_subnet_cidr_blocks[count.index]
}
output "public_subnet_ids" {
  value = aws_subnet.public[*].id
}

resource "aws_instance" "private_ec2" {
  count = length(var.private_subnet_ids)
  subnet_id = var.private_subnet_ids[count.index]
  ami           = var.ami_id
  instance_type = "t2.micro"
  tags = {
    Name = "privateEC2${count.index}}"
  }
  provisioner "local-exec" {
    command = <<EOF
echo "private EC2 ${count.index} ID is ${self.id}"
EOF
  }
}

执行结果：

$ terraform apply --auto-approve
Apply complete! Resources: 13 added, 0 changed, 0 destroyed.

所以我们需要记住的是Terraform真正看到的东西，它是一个扁平的世界，没有模块，只有资源存在。Terraform从资源中创建DAG，并且DAG中没有模块作为节点。

这就是为什么我们不能对模块使用depends_on的原因，因为在Terraform中，模块不是可以创建顶点来表示依赖关系的节点。

​

​

同步监控机制

Terraform执行多个线程，以并发方式创建资源。我们需要一个同步监控器，线程可以在此上等待，以便这些线程仅在创建依赖资源时才启动。

Terraform中的监视机制( HCL depends_on语句除外)使用创建的资源的属性(或通过本地引用)。

​

​

在module.fw_core01.firewall中，Martin展示了一个使用模块变量"vm_depends_on“的示例，该变量引用了创建的防火墙资源的属性，因此模块”示例“中的那些资源只能在创建防火墙之后才能创建。

​

​

问题

据我理解，这个问题的原因是缺少监视器，这是对当线程开始执行data "aws_subnet" "self"时应该已经创建的资源属性的引用。

由于我没有原始问题的全部来源，下面是一个再现错误的例子。

​

​

根模块

variable "vpc_cidr" {
    default = "10.5.0.0/20"
}
variable "private_subnet_cidr_blocks" {
  default = ["10.5.3.0/24","10.5.4.0/24","10.5.5.0/24"]
}

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr
}
module "private_subnet" {
  source          = "../private_subnet"
  vpc_id          = aws_vpc.this.id
  private_subnet_cidr_blocks = var.private_subnet_cidr_blocks
}
module "private_ec2" {
  source          = "../private_ec2"
  vpc_id          = aws_vpc.this.id
  private_subnet_cidr_blocks = var.private_subnet_cidr_blocks
  ami_id          = data.aws_ami.this.id
}

模块子网

variable "vpc_id" {}
variable "private_subnet_cidr_blocks" {
  type = list(string)
}

resource "aws_subnet" "private" {
  count = length(var.private_subnet_cidr_blocks)
  vpc_id     = var.vpc_id
  cidr_block = var.private_subnet_cidr_blocks[count.index]
}

模块EC2

variable "vpc_id" {}
variable "private_subnet_cidr_blocks" {
  type = list(string)
}
variable "ami_id" {}

data aws_subnet "private" {
  count      = length(var.private_subnet_cidr_blocks)
  vpc_id     = var.vpc_id
  cidr_block = var.private_subnet_cidr_blocks[count.index]
}

resource "aws_instance" "private_ec2" {
  count         = length(data.aws_subnet.private[*].id)
  subnet_id     = data.aws_subnet.private[count.index].id
  ami           = var.ami_id
  instance_type = "t2.micro"
  tags          = {
    Name = "privateEC2${count.index}}"
  }
  provisioner "local-exec" {
    command = <<EOF
echo "private EC2 ${count.index} ID is ${self.id}"
EOF
  }
}

执行结果：

$ terraform apply --auto-approve
...
Error: no matching subnet found

  on ../private_ec2/main.tf line 1, in data "aws_subnet" "private":
   1: data aws_subnet "private" {

原因

原因是使用常量变量传递私有子网CIDR，而不是使用创建的AWS子网的属性。

如果使用创建的AWS子网资源的属性，那么它作为执行data "aws_subnet" "self"的线程将等待的监视器工作。

根模块

variable "private_subnet_cidr_blocks" {
  default = ["10.5.3.0/24","10.5.4.0/24","10.5.5.0/24"]
}
module "private_ec2" {
  source          = "../private_ec2"
  vpc_id          = aws_vpc.this.id
  private_subnet_cidr_blocks = var.private_subnet_cidr_blocks # <----- Here
  ami_id          = data.aws_ami.this.id
}

由于modules中的资源与Module EC2中的资源之间不存在依赖关系，所以这两个模块中的资源创建是并行运行的。

在最初的问题中

我认为，最初问题的原因是private_subnets传递变量，而不是实际创建的AWS子网资源的属性。

module "smurf_subnet_grp" {
   source            = "../../modules/networking/subnet_grp_per_az-test"
   vpc_id            = "${module.networking_uswe2.vpc_id}"
   azs               = "${local.az_list_uswe2}"

   private_subnets   = "${var.private_subnets_uswe2}" <----- Here

}

因此，data "aws_subnet" "self"是在AWS子网正在创建或尚未创建时并发执行的。

修复

该示例的修复：

​

​

根模块

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr
}
module "private_subnet" {
  source          = "../private_subnet"
  vpc_id          = aws_vpc.this.id
  private_subnet_cidr_blocks = var.private_subnet_cidr_blocks
}
module "private_ec2" {
  source          = "../private_ec2"
  vpc_id          = aws_vpc.this.id

  #--------------------------------------------------------------------------------
  # Pass the attributes of created aws_subnet resource attributes
  #--------------------------------------------------------------------------------
  #private_subnet_cidr_blocks = var.private_subnet_cidr_blocks
  private_subnet_cidr_blocks = module.private_subnet.private_subnet_cidr_blocks  # <--- Here
  #--------------------------------------------------------------------------------

  ami_id          = data.aws_ami.this.id
}

模块子网

resource "aws_subnet" "private" {
  count = length(var.private_subnet_cidr_blocks)
  vpc_id     = var.vpc_id
  cidr_block = var.private_subnet_cidr_blocks[count.index]
}
#--------------------------------------------------------------------------------
# Output the cidr_block attributes of the AWS subnet resources created
#--------------------------------------------------------------------------------
output "private_subnet_cidr_blocks" {
  value = aws_subnet.private[*].cidr_block    # <----- Here
}

结果

$ terraform apply --auto-approve
data.aws_availability_zones.all: Refreshing state...
aws_vpc.this: Refreshing state... [id=vpc-0b338898d18a5986e]
data.aws_ami.this: Refreshing state...
data.aws_region.current: Refreshing state...
module.private_subnet.data.aws_ami.ubuntu: Refreshing state...
module.private_subnet.aws_subnet.private[2]: Refreshing state... [id=subnet-0cf916b6b9003f71f]
module.private_subnet.aws_subnet.private[1]: Refreshing state... [id=subnet-0b39beb22b23eef5d]
module.private_subnet.aws_subnet.private[0]: Refreshing state... [id=subnet-0c80c92f4023ba893]
aws_vpc.this: Creating...
aws_vpc.this: Still creating... [10s elapsed]
aws_vpc.this: Still creating... [20s elapsed]
aws_vpc.this: Creation complete after 24s [id=vpc-00069d144b5f76182]
module.private_subnet.aws_subnet.private[1]: Creating...
module.private_subnet.aws_subnet.private[2]: Creating...
module.private_subnet.aws_subnet.private[0]: Creating...
module.private_subnet.aws_subnet.private[2]: Creation complete after 5s [id=subnet-0252c6047cd56abac]
module.private_subnet.aws_subnet.private[1]: Creation complete after 6s [id=subnet-019f8cbd30db10edb]
module.private_subnet.aws_subnet.private[0]: Creation complete after 6s [id=subnet-0a1028bf17d7d81be]
module.private_ec2.data.aws_subnet.private[1]: Refreshing state...
module.private_ec2.data.aws_subnet.private[2]: Refreshing state...
module.private_ec2.data.aws_subnet.private[0]: Refreshing state...
module.private_ec2.aws_instance.private_ec2[2]: Creating...
module.private_ec2.aws_instance.private_ec2[1]: Creating...
module.private_ec2.aws_instance.private_ec2[0]: Creating...
module.private_ec2.aws_instance.private_ec2[2]: Still creating... [10s elapsed]
module.private_ec2.aws_instance.private_ec2[1]: Still creating... [10s elapsed]
module.private_ec2.aws_instance.private_ec2[0]: Still creating... [10s elapsed]
module.private_ec2.aws_instance.private_ec2[2]: Still creating... [20s elapsed]
module.private_ec2.aws_instance.private_ec2[1]: Still creating... [20s elapsed]
module.private_ec2.aws_instance.private_ec2[0]: Still creating... [20s elapsed]
module.private_ec2.aws_instance.private_ec2[2]: Still creating... [30s elapsed]
module.private_ec2.aws_instance.private_ec2[1]: Still creating... [30s elapsed]
module.private_ec2.aws_instance.private_ec2[0]: Still creating... [30s elapsed]
module.private_ec2.aws_instance.private_ec2[2]: Still creating... [40s elapsed]
module.private_ec2.aws_instance.private_ec2[1]: Still creating... [40s elapsed]
module.private_ec2.aws_instance.private_ec2[0]: Still creating... [40s elapsed]
module.private_ec2.aws_instance.private_ec2[1]: Provisioning with 'local-exec'...
module.private_ec2.aws_instance.private_ec2[1] (local-exec): Executing: ["/bin/sh" "-c" "echo \"private EC2 1 ID is i-0ced265565dfec85c\"\n"]
module.private_ec2.aws_instance.private_ec2[1] (local-exec): private EC2 1 ID is i-0ced265565dfec85c
module.private_ec2.aws_instance.private_ec2[1]: Creation complete after 46s [id=i-0ced265565dfec85c]
module.private_ec2.aws_instance.private_ec2[0]: Provisioning with 'local-exec'...
module.private_ec2.aws_instance.private_ec2[0] (local-exec): Executing: ["/bin/sh" "-c" "echo \"private EC2 0 ID is i-0f6ce62c29376c6fe\"\n"]
module.private_ec2.aws_instance.private_ec2[0] (local-exec): private EC2 0 ID is i-0f6ce62c29376c6fe
module.private_ec2.aws_instance.private_ec2[0]: Creation complete after 47s [id=i-0f6ce62c29376c6fe]
module.private_ec2.aws_instance.private_ec2[2]: Provisioning with 'local-exec'...
module.private_ec2.aws_instance.private_ec2[2] (local-exec): Executing: ["/bin/sh" "-c" "echo \"private EC2 2 ID is i-03be32b7b803eb0cc\"\n"]
module.private_ec2.aws_instance.private_ec2[2] (local-exec): private EC2 2 ID is i-03be32b7b803eb0cc
module.private_ec2.aws_instance.private_ec2[2]: Creation complete after 50s [id=i-03be32b7b803eb0cc]

Apply complete! Resources: 7 added, 0 changed, 0 destroyed.

修正原来的问题

我相信下面会解决这个问题。

module "smurf_subnet_grp" {
   source            = "../../modules/networking/subnet_grp_per_az-test"
   vpc_id            = "${module.networking_uswe2.vpc_id}"
   azs               = "${local.az_list_uswe2}"

   #--------------------------------------------------------------------------------
   # Pass the cidr_block attribute of aws_subnet resource created in module.networking_uswe2
   #------------------------------------------------------------------------------   
   #private_subnets   = "${var.private_subnets_uswe2}"
   private_subnets   = module.networking_uswe2.private_subnet_cidr_blocks  # <---- Here
   #------------------------------------------------------------------------------   
}