使用Go Task的Yaml Azure管道发出安装Terratest -在子文件夹中发出触发terratest测试

Question

我在安装天蓝色yaml管道时遇到了这个问题：

C:\hostedtoolcache\windows\go\1.17.1\x64\bin\go.exe install -v github.com/gruntwork-io/terratest@v0.40.6
go: downloading github.com/gruntwork-io/terratest v0.40.6
go install: github.com/gruntwork-io/terratest@v0.40.6: module github.com/gruntwork-io/terratest@v0.40.6 found, but does not contain package github.com/gruntwork-io/terratest
##[error]The Go task failed with an error: Error: The process 'C:\hostedtoolcache\windows\go\1.17.1\x64\bin\go.exe' failed with exit code 1
Finishing: Install Go Terratest module - v0.40.6

我的安装代码如下：

            - task: Go@0
              displayName: Install Go Terratest module - v$(TERRATEST_VERSION)
              inputs:
                command: custom
                customCommand: install
                arguments: $(TF_LOG) github.com/gruntwork-io/terratest@v$(TERRATEST_VERSION)
                workingDirectory: $(pipeline_artefact_folder_extract)/$(pathToTerraformRootModule)

但我在使用竖琴时犯了错误。

Bellow是我的代码树的屏幕截图：

​

​

我在(例如) Terraform\azure_v2_X\ResourceModules子目录中有terraform代码，在Terraform\azure_v2_X\ResourceModules子目录中有terratest测试(在app_configuration resourceModules的屏幕截图app_configuration测试中)。

在我最迟的模块中，我调用我的resourceModule，如下代码所示：

######test in a un isolated Resource Group defined in locals
module "app_configuration_tobetested" {    
    source = "../../ResourceModules/app_configuration"
    resource_group_name = local.rg_name
    location = local.location
    environment = var.ENVIRONMENT
    sku = "standard"
    // rem : here app_service_shared prefix and app_config_shared prefix are the same !
    app_service_prefix = module.app_configuration_list_fortests.settings.frontEnd_prefix
#    stage = var.STAGE
    app_config_list = module.app_configuration_list_fortests.settings.list_app_config
}

在Go文件中，我测试我的模块结果是否符合我想要的预期结果：

package RM_app_configuration_Test

import (
    "os"
    "testing"

    //  "github.com/gruntwork-io/terratest/modules/azure"
    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

var (
    globalBackendConf = make(map[string]interface{})
    globalEnvVars     = make(map[string]string)
)

func TestTerraform_RM_app_configuration(t *testing.T) {
    t.Parallel()

    // terraform Directory
    fixtureFolder := "./"

    // backend specification
    strlocal := "RMapCfg_"

    // input value
    inputStage       := "sbx_we"
    inputEnvironment := "SBX"
    inputApplication := "DEMO"

    // expected value
    expectedRsgName := "z-adf-ftnd-shrd-sbx-ew1-rgp01"
    //  expectedAppCfgPrefix := "z-adf-ftnd-shrd"
    expectedAppConfigReader_ID := "[/subscriptions/f04c8fd5-d013-41c3-9102-43b25880d2e2/resourceGroups/z-adf-ftnd-shrd-sbx-ew1-rgp01/providers/Microsoft.AppConfiguration/configurationStores/z-adf-ftnd-shrd-sbx-ew1-blue-sbx-cfg01 /subscriptions/f04c8fd5-d013-41c3-9102-43b25880d2e2/resourceGroups/z-adf-ftnd-shrd-sbx-ew1-rgp01/providers/Microsoft.AppConfiguration/configurationStores/z-adf-ftnd-shrd-sbx-ew1-green-sbx-cfg01]"

    // getting enVars from environment variables
    /*
        Go and Terraform uses two differents methods for Azure authentification.
        ** Terraform authentification is explained bellow :
        - https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret#configuring-the-service-principal-in-terraform
        ** Go authentification is explained bellow
        - https://learn.microsoft.com/en-us/azure/developer/go/azure-sdk-authorization#use-environment-based-authentication

        ** Terratest is using both authentification methods regarding the work it has to be done :
        - azure existences tests uses Go azure authentification :
            - https://github.com/gruntwork-io/terratest/blob/master/modules/azure/authorizer.go#L11
        - terraform commands uses terraform authentification :
            - https://github.com/gruntwork-io/terratest/blob/0d654bd2ab781a52e495f61230cf892dfba9731b/modules/terraform/cmd.go#L12
            - https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_client_secret#configuring-the-service-principal-in-terraform
        so both authentification methods have to be implemented
    */
    // getting terraform EnvVars from Azure Go environment variables
    ARM_CLIENT_ID := os.Getenv("AZURE_CLIENT_ID")
    ARM_CLIENT_SECRET := os.Getenv("AZURE_CLIENT_SECRET")
    ARM_TENANT_ID := os.Getenv("AZURE_TENANT_ID")
    ARM_SUBSCRIPTION_ID := os.Getenv("ARM_SUBSCRIPTION_ID")

    if ARM_CLIENT_ID != "" {
        globalEnvVars["ARM_CLIENT_ID"] = ARM_CLIENT_ID
        globalEnvVars["ARM_CLIENT_SECRET"] = ARM_CLIENT_SECRET
        globalEnvVars["ARM_SUBSCRIPTION_ID"] = ARM_SUBSCRIPTION_ID
        globalEnvVars["ARM_TENANT_ID"] = ARM_TENANT_ID
    }

    // getting terraform backend from environment variables
    resource_group_name := os.Getenv("resource_group_name")
    storage_account_name := os.Getenv("storage_account_name")
    container_name := os.Getenv("container_name")
    key := strlocal + os.Getenv("key")

    if resource_group_name != "" {
        globalBackendConf["resource_group_name"] = resource_group_name
        globalBackendConf["storage_account_name"] = storage_account_name
        globalBackendConf["container_name"] = container_name
        globalBackendConf["key"] = key
    }

    // User Terratest to deploy the infrastructure
    terraformOptions := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
        // website::tag::1::Set the path to the Terraform code that will be tested.
        // The path to where our Terraform code is located
        TerraformDir: fixtureFolder,
        // Variables to pass to our Terraform code using -var options
        Vars: map[string]interface{}{
            "STAGE":       inputStage,
            "ENVIRONMENT": inputEnvironment,
            "APPLICATION": inputApplication,
        },

        EnvVars: globalEnvVars,

        // backend values to set when initialziing Terraform
        BackendConfig: globalBackendConf,

        // Disable colors in Terraform commands so its easier to parse stdout/stderr
        NoColor: true,
    })

    // website::tag::4::Clean up resources with "terraform destroy". Using "defer" runs the command at the end of the test, whether the test succeeds or fails.
    // At the end of the test, run `terraform destroy` to clean up any resources that were created
    defer terraform.Destroy(t, terraformOptions)

    // website::tag::2::Run "terraform init" and "terraform apply".
    // This will run `terraform init` and `terraform apply` and fail the test if there are any errors
    terraform.InitAndApply(t, terraformOptions)

    // tests the resource_group for the app_configuration
    /*
        actualAppConfigReaderPrefix := terraform.Output(t, terraformOptions, "app_configuration_tested_prefix")
        assert.Equal(t, expectedAppCfgprefix, actualAppConfigReaderPrefix)
    */
    actualRSGReaderName := terraform.Output(t, terraformOptions, "app_configuration_tested_RG_name")
    assert.Equal(t, expectedRsgName, actualRSGReaderName)

    actualAppConfigReader_ID := terraform.Output(t, terraformOptions, "app_configuration_tobetested_id")
    assert.Equal(t, expectedAppConfigReader_ID, actualAppConfigReader_ID)
}

事实上，在本地，我可以从我的主文件夹Terraform\Azure_v2_X\Tests_Unit_ResourceModules执行以下命令来触发raw中的所有测试：

• (来自Go
• )

Go test ./...

使用Go版本1.12，我可以将GO111MODULE=auto设置为具有相同的结果。但是对于Go 1.17，我现在必须设置GO111MODULE=off来触发我的测试。

现在，我有两个主要问题困扰着我:

1. How我可以从蔚蓝管道?

导入Terratest (和其他)模块吗？

1. What我必须做正确的Go模块和terratest ?我在我的主文件夹_Terraform\Azure_v2_X\Tests_Unit_ResourceModules_中没有Go代码，并且想要在Azure管道中的一个简单的命令行中触发所有的sub_folder go测试。

谢谢你的帮助。

诚挚的问候,

Answer 1

我会再次回答我自己的问题。:D

因此，就目前而言，使用以下版本：

• -- GOVERSION: 1.17.1
• -- TERRAFORM_VERSION :1.1.7
• - TERRATEST_VERSION: 0.40.6

文件夹层次结构更改了以下内容，涉及最迟的测试：

​

​

--我不再尝试导入我的Terratest模块了。显然，上面的(所以第1点)是有问题的。

我现在只需要：

1. Go使用脚本

逐个分别对我最迟的

• 中的每一个

所以我的管道变成了：

    - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
      displayName: Install Terraform $(TERRAFORM_VERSION)
      inputs:
         terraformVersion: $(TERRAFORM_VERSION)

    - task: GoTool@0
      displayName: 'Use Go $(GOVERSION)'
      inputs:
        version: $(GOVERSION)
        goPath: $(GOPATH)
        goBin: $(GOBIN)

    - task: PowerShell@2
      displayName: run Terratest for $(pathToTerraformRootModule)
      inputs:
        targettype : 'filePath'
        filePath: $(pipeline_artefact_folder_extract)/$(pathToTerraformRootModule)/$(Run_Terratest_script)
        workingDirectory: $(pipeline_artefact_folder_extract)/$(pathToTerraformRootModule)
      env:
        # see https://learn.microsoft.com/en-us/azure/developer/go/azure-sdk-authorization#use-environment-based-authentication
        # for Azure authentification with Go
        ARM_SUBSCRIPTION_ID: $(TF_VAR_ARM_SUBSCRIPTION_ID)
        AZURE_CLIENT_ID: $(TF_VAR_ARM_CLIENT_ID)
        AZURE_TENANT_ID: $(TF_VAR_ARM_TENANT_ID)
        AZURE_CLIENT_SECRET: $(TF_VAR_ARM_CLIENT_SECRET) # set as pipeline secret
        resource_group_name: $(storageAccountResourceGroup)
        storage_account_name: $(storageAccount)
        container_name: $(stateBlobContainer)
        key: '$(MODULE)-$(TF_VAR_APPLICATION)-$(TF_VAR_ENVIRONMENT).tfstate'
        GO111MODULE: 'auto'

在我的terratest子文件夹的主文件夹中，我的run_terratests.ps1脚本和terratest列表文件如下所示：

run_terratests.ps1

# this file is based on https://github.com/google/go-cloud/blob/master/internal/testing/runchecks.sh
#
# This script runs all go Terratest suites,
# compatibility checks, consistency checks, Wire, etc.

$moduleListFile = "./Terratests"
# regex to filter : not began with #
$regexFilter = "^[^#]"

# read the ModuleListFile
[object] $arrayFromFile = Get-Content -Path $moduleListFile | Where-Object { $_ -match $regexFilter} | ConvertFrom-String -PropertyNames folder, totest

$result = 0 # set no error by default
# get the actual folder 
$main_path = Get-Location | select -ExpandProperty "Path"
#read the array to show if to be tested !
foreach ($line in $arrayFromFile) {
    # write-Host $line
    if ($line.totest -eq "yes") {
        $path = $line.folder
        set-location $main_path\$path
        $myPath = Get-Location
        # Write-Host $myPath
        # trigger terratest for files
        Go test ./...
    }
    if ($false -eq $?)
    {
        $result = 1
    }    
}
# back to school :D
set-location $main_path

if ($result -eq 1)
{
    Write-Error "Msbuild exit code indicate test failure."
    Write-Host "##vso[task.logissue type=error]Msbuild exit code indicate test failure."
    exit(1)
}

代码

  if ($false -eq $?)
    {
        $result = 1
    }  

在不逃避其他测试的情况下，使管道在测试错误时失败是有用的。

Terratests

# this file lists all the modules to be tested in the "Tests_Unit_ConfigHelpers" repository.
# it us used by the "run_terratest.ps1" powershell script to trigger terratest for each test.
#
# Any line that doesn't begin with a '#' character and isn't empty is treated
# as a path relative to the top of the repository that has a module in it.
# The 'tobetested' field specifies whether this is a module that have to be tested.
#
# this file is based on https://github.com/google/go-cloud/blob/master/allmodules

# module-directory              tobetested
azure_constants                     yes
configure_app_srv_etc               yes
configure_frontdoor_etc             yes
configure_hostnames                 yes
constants                           yes
FrontEnd_AppService_slots/_main     yes
FrontEnd_AppService_slots/settings  yes
merge_maps_of_strings               yes
name                                yes
name_template                       yes
network/hostname_generator          yes
network/hostnames_generator         yes
replace_2vars_into_string_etc       yes
replace_var_into_string_etc         yes
sorting_map_with_an_other_map       yes

每个terratest文件夹中的更改是，我将添加go.mod和go.sum文件：

$ go mod init mytest
go: creating new go.mod: module mytest
go: to add module requirements and sums:
go mod tidy

和

$ go mod tidy
# link each of the go modules needed for your terratest module

这样，powershell脚本中的go test ./...将降低所需的go模块，并为该特殊测试运行测试。

谢谢你的阅读和投票，如果你认为这有帮助的话:)