如何为其他(非TBC)作业导出作用域变量？

Question

感谢您的工作。我正在使用您的模板处理Terraform，并且需要根据不同的环境，在所包含的作业中，以及在其他的和我的作业中，不同地设置我的CUSTOM_ENVIRONMENT_NAME变量。

有不同的逻辑来区分我的管道上的环境：

• 您的逻辑基于分支，但生产和暂存都共享主分支。
• 基于gitlab项目的Orange作业(一个项目=一个环境)
• 我自己的工作，我还没有精确的逻辑

到目前为止，我发现了不同的可能机制：

• 使用$CI_ENVIRONMENT_NAME --但我需要设置环境:每个作业的名称: foo，我包含了许多我不想覆盖的不同的任务
• 使用$CI_COMMIT_REF_NAME --但是在模板中，生产和暂存共享相同的分支
• 在预先录制的作业中导出工件，参见https://docs.gitlab.com/ee/ci/variables/#pass-an-environment-variable-to-another-job
• 使用extends关键字复制作业并手动设置CI_ENVIRONMENT_NAME或CUSTOM_ENVIRONMENT_NAME
• 使用您的作用域变量语法-但它只适用于您的作业。

我想使用两个组合:使用作用域变量并使用工件将它们导出到其他作业。

有什么方法可以导出作用域变量(cf.( https://docs.gitlab.com/ee/ci/variables/#pass-an-environment-variable-to-another-job)来自tflint作业(它正在我的管道开始运行)？

下面是一个示例gitlab-ci文件：

include:
  # terraform
  - project: 'to-be-continuous/terraform' # template principal de TBC
    ref: '2.3.0'
    file: '/templates/gitlab-ci-terraform.yml'
  - project: 'to-be-continuous/custom/devops-store' # template additionnel pour utiliser les runners RSC plutôt que DIOD/GIN
    ref: '1.1.2'
    file: '/templates/gitlab-ci-terraform-dos.yml'

  # PDN
  - project: "dixsiptal/newdelivery/templates/common"
    ref: "1.0.30"
    file: "templates/gitlab-ci-ansible.yml"
  - project: "dixsiptal/newdelivery/templates/common-token"
    ref: "1.0.1"
    file: "settoken.yml"
  - project: "dixsiptal/newdelivery/templates/pli/pdn"
    ref: "2.1.2"
    file: "templates/gitlab-ci.yml"

  # PLI docker
  # NB : common et common-token sont également des pré-requis au PDN
  - project: "dixsiptal/newdelivery/templates/pli/docker"
    ref: "3.1.0"
    file: "templates/gitlab-ci.yml"


variables:
  # les lignes scoped suivantes permettent de définir ENV_NOMMAGE_IPANEMA différemment selon l'environnement, grâce au template TBC
  # ENV_NOMMAGE_IPANEMA sera alors utilisable depuis tous les jobs
  # cf. https://to-be-continuous.gitlab.io/doc/usage/#scoped-variables
  scoped__ENV_NOMMAGE_IPANEMA__if__CI_ENVIRONMENT_NAME__equals__production: production
  scoped__ENV_NOMMAGE_IPANEMA__if__CI_ENVIRONMENT_NAME__equals__staging: maintenance
  scoped__ENV_NOMMAGE_IPANEMA__if__CI_ENVIRONMENT_NAME__equals__integration: e2e
  scoped__ENV_NOMMAGE_IPANEMA__if__CI_ENVIRONMENT_NAME__startswith__review: review

  # For the docker and the PDN template:
  INVENTORY: "./inventories/${ENV_NOMMAGE_IPANEMA}/terraform_inventory.ini"
  # or, if it is not possible to interpolate ENV_NOMMAGE_IPANEMA at this stage:
  # scoped__INVENTORY__if__CI_ENVIRONMENT_NAME__equals__production: "inventories/production/terraform_inventory.ini"
  # scoped__INVENTORY__if__CI_ENVIRONMENT_NAME__equals__staging: "inventories/maintenance/terraform_inventory.ini"
  # scoped__INVENTORY__if__CI_ENVIRONMENT_NAME__equals__integration: "inventories/e2e/terraform_inventory.ini"
  # scoped__INVENTORY__if__CI_ENVIRONMENT_NAME__startswith__review: "inventories/review/terraform_inventory.ini"

  # for the PDN template:
  PDN_PROJET: "${CI_SERVER_HOST}/wanctr/sandbox/iac/pdn_inventories/pdn_inventory_${ENV_NOMMAGE_IPANEMA}.git" # doit être différencié selon l'environnement. Imposé par le template PDN
  # same thing would apply here
  # scoped__PDN_PROJET__if__CI_ENVIRONMENT_NAME__equals__integration: "${CI_SERVER_HOST}/wanctr/sandbox/iac/pdn_inventories/pdn_inventory_e2e.git"
  # scoped__PDN_PROJET__if__CI_ENVIRONMENT_NAME__startswith__review: "${CI_SERVER_HOST}/wanctr/sandbox/iac/pdn_inventories/pdn_inventory_review.git"
  # etc.

  # terraform
  TF_PROJECT_DIR: "./terraform"
  TF_OUTPUT_DIR: "../ansible/inventories" # relatif à TF_PROJECT_DIR
  TF_BINARY_VERSION: "1.0.10"
  TF_BRMC_PROVIDER_VERSION: "2.8.1_2.7.0" # peut avoir la forme v1_v2 par exemple 2.8.1_2.7.0 pour avoir 2 versions de providers simultanément utile pour upgrade) (cf. la liste des versions disponibles sur https://gitlab.tech.orange/wanctr/sandbox/iac/terrabrmc/container_registry/18687)
  TF_IMAGE: "registry.${CI_SERVER_HOST}/wanctr/sandbox/iac/terrabrmc:${TF_BINARY_VERSION}-${TF_BRMC_PROVIDER_VERSION}"
  #TF_EXTRA_OPTS: ""
  TF_INIT_OPTS: "-upgrade" # pour upgrade la version du provider le cas échéant

  TF_REVIEW_ENABLED: "true" # permet l'utilisation des envs de test, créées automatiquement aux push sur des branches autres que "master" et "develop"
  TF_REVIEW_EXTRA_OPTS: "-var-file=values-review.tfvars"

  TF_INTEG_ENABLED: "true" # e2e/qualif
  TF_INTEG_EXTRA_OPTS: "-var-file=values-e2e.tfvars"

  TF_STAGING_ENABLED: "true" # maintenance
  TF_STAGING_EXTRA_OPTS: "-var-file=values-maintenance.tfvars"

  TF_PROD_ENABLED: "true" # production
  TF_PROD_EXTRA_OPTS: "-var-file=values-production.tfvars"
  
  BRMC_HOST: "https://brmc.si.fr.intraorange"
  BRMC_TENANT: "vsphere.local"
  BRMC_USERNAME: "$BRMC_API_ACCOUNT_USERNAME"
  BRMC_PASSWORD: "$BRMC_API_ACCOUNT_PASSWORD"

  # ansible utilisé par templates communs DESI https://gitlab.tech.orange/dixsiptal/newdelivery/templates/common/-/blob/1.0.30/README.md
  ANSIBLE_PATH: "./ansible" # chemin vers les ressources ansible
  ANSIBLE_USER: "ansible"
  ODE_ANSIBLE_VERSION: "2.9" # ou 2.10 ?

  # docker https://gitlab.tech.orange/dixsiptal/newdelivery/templates/pli/docker/-/blob/3.1.0/README.md
  # INVENTORY is used here
  .ansible:
    extends: .ansible-runner-ode # indique au template gitlab-ci du PLI docker d'utiliser ODE pour le déploiement
  DOCKER_INVENTORY_SECTION: "cluster"
  DOCKER_REDHAT_VG: "docker_vg"
  # etc.

  # PLI PostgreSQL
  # ...

  # ODE CLI
  ODE_CLI_VERSION: "2.1.8" # ou "2.3.0" ?

  # custom-traefik (one of my own)
  ANSIBLE_REQUIREMENTS: "${ANSIBLE_PATH}/requirements.yml"
  # ...
  CONFIG_APPLICATIVE_MAIN_INVENTORY_DIR: "config_applicative/inventories/${ENV_NOMMAGE_IPANEMA}"
  # ...


pre-test-scoped: # testing the dotenv artifact variable setting
  stage: test
  rules:
  - when: always
  script:
    - echo "ENV_NOMMAGE_IPANEMA_envfile=value_from_other_job" >> justtesting.env
  artifacts:
    reports:
      dotenv: justtesting.env

# echoing preset variables for testing
test-scoped:
  stage: test
  image: dockerfactory.tech.orange/ode:$ODE_CLI_VERSION
  variables:
    ENV_NOMMAGE_IPANEMA_rules_by_commitrefname: "review" # valeur par défaut
  environment: # Ideally I would like not to indicate the env name each time for concision and simplicity when importing templates
    name: testscoped
  needs: ["pre-test-scoped"]
  rules:
  - when: on_success
  - if: $CI_COMMIT_REF_NAME == "master"
    variables:
      ENV_NOMMAGE_IPANEMA_rules_by_commitrefname: "production" # et maintenance ?..
  - if: $CI_COMMIT_REF_NAME == "develop"
    variables:
      ENV_NOMMAGE_IPANEMA_rules_by_commitrefname: "E2E"
  - if: $CI_COMMIT_REF_NAME == "scoped-variables"
    variables:
      ENV_NOMMAGE_IPANEMA_rules_by_commitrefname: "scoped-variables-review-test"

  - if: $CI_ENVIRONMENT_NAME == "production"
    variables:
      ENV_NOMMAGE_IPANEMA_rules_by_cienvname: "production" # et maintenance ?..
  - if: $CI_ENVIRONMENT_NAME == "staging"
    variables:
      ENV_NOMMAGE_IPANEMA_rules_by_cienvname: "maintenance"
  - if: $CI_ENVIRONMENT_NAME == "integration"
    variables:
      ENV_NOMMAGE_IPANEMA_rules_by_cienvname: "E2E" # et maintenance ?..
  - if: $CI_ENVIRONMENT_NAME == "review"
    variables:
      ENV_NOMMAGE_IPANEMA_rules_by_cienvname: "review"

  script:
    - echo $CI_COMMIT_REF_NAME
    - echo $CI_ENVIRONMENT_NAME
    - echo $ENV_NOMMAGE_IPANEMA_tfscoped
    - echo $ENV_NOMMAGE_IPANEMA_rules_by_commitrefname
    - echo $ENV_NOMMAGE_IPANEMA_rules_by_cienvname
    - echo $ENV_NOMMAGE_IPANEMA_projectenvset # set in the gitlab project with a scope
    - echo $ENV_NOMMAGE_IPANEMA_envfile


custom-traefik: # job supplémentaire
  image: dockerfactory.tech.orange/ode:$ODE_CLI_VERSION
  when: manual
  stage: "Gestion"

  script:
  # ...
  # this job uses scoped varibles

export-inventory-file:
  image: dockerfactory.tech.orange/ode:$ODE_CLI_VERSION
  when: manual
  stage: "Gestion"

  script:
  # ...
  # this job uses scoped varibles

Answer 1

当然，您可以在您的作用域变量中使用以下代码来模拟连续的.gitlab-ci.yml特性

.base-scripts: &base-scripts |
  set -e

  function log_info() {
      echo -e "[\\e[1;94mINFO\\e[0m] $*"
  }

  function log_warn() {
      echo -e "[\\e[1;93mWARN\\e[0m] $*"
  }

  function log_error() {
      echo -e "[\\e[1;91mERROR\\e[0m] $*"
  }

  function unscope_variables() {
    _scoped_vars=$(env | awk -F '=' "/^scoped__[a-zA-Z0-9_]+=/ {print \$1}" | sort)
    if [[ -z "$_scoped_vars" ]]; then return; fi
    log_info "Processing scoped variables..."
    for _scoped_var in $_scoped_vars
    do
      _fields=${_scoped_var//__/:}
      _condition=$(echo "$_fields" | cut -d: -f3)
      case "$_condition" in
      if) _not="";;
      ifnot) _not=1;;
      *)
        log_warn "... unrecognized condition \\e[1;91m$_condition\\e[0m in \\e[33;1m${_scoped_var}\\e[0m"
        continue
      ;;
      esac
      _target_var=$(echo "$_fields" | cut -d: -f2)
      _cond_var=$(echo "$_fields" | cut -d: -f4)
      _cond_val=$(eval echo "\$${_cond_var}")
      _test_op=$(echo "$_fields" | cut -d: -f5)
      case "$_test_op" in
      defined)
        if [[ -z "$_not" ]] && [[ -z "$_cond_val" ]]; then continue; 
        elif [[ "$_not" ]] && [[ "$_cond_val" ]]; then continue; 
        fi
        ;;
      equals|startswith|endswith|contains|in|equals_ic|startswith_ic|endswith_ic|contains_ic|in_ic)
        # comparison operator
        # sluggify actual value
        _cond_val=$(echo "$_cond_val" | tr '[:punct:]' '_')
        # retrieve comparison value
        _cmp_val_prefix="scoped__${_target_var}__${_condition}__${_cond_var}__${_test_op}__"
        _cmp_val=${_scoped_var#"$_cmp_val_prefix"}
        # manage 'ignore case'
        if [[ "$_test_op" == *_ic ]]
        then
          # lowercase everything
          _cond_val=$(echo "$_cond_val" | tr '[:upper:]' '[:lower:]')
          _cmp_val=$(echo "$_cmp_val" | tr '[:upper:]' '[:lower:]')
        fi
        case "$_test_op" in
        equals*)
          if [[ -z "$_not" ]] && [[ "$_cond_val" != "$_cmp_val" ]]; then continue; 
          elif [[ "$_not" ]] && [[ "$_cond_val" == "$_cmp_val" ]]; then continue; 
          fi
          ;;
        startswith*)
          if [[ -z "$_not" ]] && [[ "$_cond_val" != "$_cmp_val"* ]]; then continue; 
          elif [[ "$_not" ]] && [[ "$_cond_val" == "$_cmp_val"* ]]; then continue; 
          fi
          ;;
        endswith*)
          if [[ -z "$_not" ]] && [[ "$_cond_val" != *"$_cmp_val" ]]; then continue; 
          elif [[ "$_not" ]] && [[ "$_cond_val" == *"$_cmp_val" ]]; then continue; 
          fi
          ;;
        contains*)
          if [[ -z "$_not" ]] && [[ "$_cond_val" != *"$_cmp_val"* ]]; then continue; 
          elif [[ "$_not" ]] && [[ "$_cond_val" == *"$_cmp_val"* ]]; then continue; 
          fi
          ;;
        in*)
          if [[ -z "$_not" ]] && [[ "__${_cmp_val}__" != *"__${_cond_val}__"* ]]; then continue; 
          elif [[ "$_not" ]] && [[ "__${_cmp_val}__" == *"__${_cond_val}__"* ]]; then continue; 
          fi
          ;;
        esac
        ;;
      *)
        log_warn "... unrecognized test operator \\e[1;91m${_test_op}\\e[0m in \\e[33;1m${_scoped_var}\\e[0m"
        continue
        ;;
      esac
      # matches
      _val=$(eval echo "\$${_target_var}")
      log_info "... apply \\e[32m${_target_var}\\e[0m from \\e[32m\$${_scoped_var}\\e[0m${_val:+ (\\e[33;1moverwrite\\e[0m)}"
      _val=$(eval echo "\$${_scoped_var}")
      export "${_target_var}"="${_val}"
    done
    log_info "... done"
  }


# Generic base job
.base-job:
  before_script:
    - *base-scripts
    - unscope_variables

variables:
  SOME_VAR: "this is the default value"
  scoped__SOME_VAR__if__CI_JOB_NAME__equals__job1: "this is value for job#1"
  scoped__SOME_VAR__if__CI_JOB_NAME__equals__job2: "this is value for job#2"

# Then have all your jobs inherit from .base-job
job1:
  extends: .base-job
  stage: stage1
  script:
    - echo "$SOME_VAR"

job2:
  extends: .base-job
  stage: stage2
  script:
    - echo "$SOME_VAR"

所有的魔术都在unscope_variables()函数中实现。

/!\现在是一些警告

1. 对$INVENTORY和$PDN_PROJET的后期评估(都在值中使用作用域变量$ENV_NOMMAGE_IPANEMA )不能运行
2. 在您的作用域变量中使用$CI_ENVIRONMENT_NAME作为测试条件只适用于具有定义环境的作业(此值在其他地方未设置)。您可能最好使用$environment_type变量，由Terraform作业设置，并作为dotenv伪影传播到下游管道