terraform计划使用terraform cloud后端在每次运行时重新创建资源

Question

我遇到了一个问题，terraform plan重新创建了不需要每次运行都重新创建的资源。这是一个问题，因为一些步骤依赖于可用的资源，并且由于每次运行都会重新创建这些资源，因此脚本无法完成。

我的设置是Github Actions，Linode LKE，Terraform Cloud。

我的main.tf文件如下所示：

terraform {
  required_providers {
    linode = {
      source  = "linode/linode"
      version = "=1.16.0"
    }
    helm = {
      source = "hashicorp/helm"
      version = "=2.1.0"
    }
  }
  backend "remote" {
    hostname      = "app.terraform.io"
    organization  = "MY-ORG-HERE"
    workspaces {
      name = "MY-WORKSPACE-HERE"
    }    
  }
}

provider "linode" {
}

provider "helm" {
  debug   = true
  kubernetes {
    config_path = "${local_file.kubeconfig.filename}"
  }
}

resource "linode_lke_cluster" "lke_cluster" {
    label       = "MY-LABEL-HERE"
    k8s_version = "1.21"
    region      = "us-central"

    pool {
        type  = "g6-standard-2"
        count = 3
    }
}

和我的outputs.tf文件

resource "local_file" "kubeconfig" {
  depends_on   = [linode_lke_cluster.lke_cluster]
  filename     = "kube-config"
  # filename     = "${path.cwd}/kubeconfig"
  content      = base64decode(linode_lke_cluster.lke_cluster.kubeconfig)
}

resource "helm_release" "ingress-nginx" {
  # depends_on   = [local_file.kubeconfig]
  depends_on = [linode_lke_cluster.lke_cluster, local_file.kubeconfig]
  name       = "ingress"
  repository = "https://kubernetes.github.io/ingress-nginx"
  chart      = "ingress-nginx"
}

resource "null_resource" "custom" {
  depends_on   = [helm_release.ingress-nginx]
  # change trigger to run every time
  triggers = {
    build_number = "${timestamp()}"
  }

  # download kubectl
  provisioner "local-exec" {
    command = "curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl"
  }

  # apply changes
  provisioner "local-exec" {
    command = "./kubectl apply -f ./k8s/ --kubeconfig ${local_file.kubeconfig.filename}"
  }
}

在Github Actions中，我运行以下步骤：

jobs:
  init-terraform:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: ./terraform
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
        with:
          ref: 'privatebeta-kubes'
      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1
        with:
          cli_config_credentials_token: ${{ secrets.TERRAFORM_API_TOKEN }}
      - name: Terraform Init
        run: terraform init
      - name: Terraform Format Check
        run: terraform fmt -check -v
      - name: List terraform state
        run: terraform state list
      - name: Terraform Plan
        run: terraform plan
        id: plan
        env:
          LINODE_TOKEN: ${{ secrets.LINODE_TOKEN }}

当我查看terraform state list的结果时，我可以看到我的资源：

Run terraform state list
  terraform state list
  shell: /usr/bin/bash -e {0}
  env:
    TERRAFORM_CLI_PATH: /home/runner/work/_temp/3f9749b8-515b-4cb4-8053-1a6318496321
/home/runner/work/_temp/3f9749b8-515b-4cb4-8053-1a6318496321/terraform-bin state list
helm_release.ingress-nginx
linode_lke_cluster.lke_cluster
local_file.kubeconfig
null_resource.custom

但是我的terraform plan失败了，问题似乎源于这些资源试图重新创建。

Run terraform plan
  terraform plan
  shell: /usr/bin/bash -e {0}
  env:
    TERRAFORM_CLI_PATH: /home/runner/work/_temp/3f9749b8-515b-4cb4-8053-1a6318496321
    LINODE_TOKEN: ***
/home/runner/work/_temp/3f9749b8-515b-4cb4-8053-1a6318496321/terraform-bin plan
Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...
Waiting for the plan to start...

Terraform v1.0.2
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...
linode_lke_cluster.lke_cluster: Refreshing state... [id=31946]
local_file.kubeconfig: Refreshing state... [id=fbb5520298c7c824a8069397ef179e1bc971adde]
helm_release.ingress-nginx: Refreshing state... [id=ingress]
╷
│ Error: Kubernetes cluster unreachable: stat kube-config: no such file or directory
│ 
│   with helm_release.ingress-nginx,
│   on outputs.tf line 8, in resource "helm_release" "ingress-nginx":
│    8: resource "helm_release" "ingress-nginx" {

有没有办法告诉terraform它不需要重新创建这些资源？

Answer 1

关于实际显示的错误，错误: Kubernetes cluster unreachable: stat kibe-：no the file or directory config...它正在引用您的输出文件...我找到了这个可以帮助你纠正特定错误的代码：https://github.com/hashicorp/terraform-provider-helm/issues/418

另一件事对我来说很奇怪。为什么你的outputs.tf指的是“资源”而不是“输出”。你的outputs.tf不应该是这样的吗？

output "local_file_kubeconfig" {
  value = "reference.to.resource"
}

我还看到你的状态文件/后端配置看起来像是正确配置的。

我建议您登录到您的terraform云帐户，以验证工作空间确实存在，正如预期的那样。它是告诉terraform不要重新创建它管理的资源的状态文件。

如果资源已经存在并且terraform正在尝试重新创建它们，这可能表明这些资源是在使用terraform之前创建的，或者可能是在另一个terraform云工作空间或计划中创建的。

您是否在使用此计划的任何时候重命名了您的后端工作空间？我指的是您的main.tf文件，这部分内容是MY-WORKSPACE-HERE：

terraform {
  required_providers {
    linode = {
      source  = "linode/linode"
      version = "=1.16.0"
    }
    helm = {
      source = "hashicorp/helm"
      version = "=2.1.0"
    }
  }
  backend "remote" {
    hostname      = "app.terraform.io"
    organization  = "MY-ORG-HERE"
    workspaces {
      name = "MY-WORKSPACE-HERE"
    }    
  }
}

不幸的是，我不是kurbenetes专家，所以可能需要更多的帮助。