在后续阶段对gl- SAST -report.json工件的访问

Question

我想在CI的后续阶段使用在SAST过程中创建的gl report.json文件，但是找不到它。

ci.yml

include:
  - template: Security/SAST.gitlab-ci.yml

stages:
  - test
  - .post
sast:
  rules:
    - if: $CI_COMMIT_TAG

send-reports:
  stage: .post
  dependencies: 
    - sast
  script: 
    - ls
    - echo "in post stage"
    - cat gl-sast-report.json

输出：

Running with gitlab-runner 13.2.1 (efa30e33)
on blah blah blah
Preparing the "docker" executor
00:01
.
.
.

Preparing environment
00:01
Running on runner-zqk9bcef-project-4296-concurrent-0 via ff93ba7b6ee2...
Getting source from Git repository
00:01
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in blah blah
Checking out 9c2edf67 as 39-test-dso...
Removing gl-sast-report.json
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:03
$ ls
<stuff in the repo>
$ echo "in .post stage"
in post stage
$ cat gl-sast-report.json
cat: can't open 'gl-sast-report.json': No such file or directory
ERROR: Job failed: exit code 1

您可以看到Removing gl-sast-report.json行，我认为这就是问题所在。

在https://gitlab.com/gitlab-org/gitlab/-/blob/v11.11.0-rc2-ee/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml#L33-45的SAST.gitlab-ci.yml中，我没有看到这一点。

对于如何在CI管道的下一阶段使用这个工件有什么想法吗？

更新：

因此，我在下面尝试了k33g_org的建议，但没有结果。这似乎是由于sast模板的限制，特别是。做了下面的测试。

include:
  - template: Security/SAST.gitlab-ci.yml

stages:
  - test
  - upload

something:
  stage: test
  script:
      - echo "in something"
      - echo "this is something" > something.txt
  artifacts:
      paths: [something.txt]

sast:
  before_script:
      - echo "hello from before sast"
      - echo "this is in the file" > test.txt
  artifacts:
    reports:
      sast: gl-sast-report.json
    paths: [gl-sast-report.json, test.txt]

send-reports:
  stage: upload
  dependencies:
    - sast
    - something
  before_script:
      - echo "This is the send-reports before_script"
  script:
    - echo "in send-reports job"
    - ls
  artifacts:
      reports:
          sast: gl-sast-report.json

三项变动：

1. 用k33g_org的建议更新代码
2. 在sast作业中创建了另一个工件(查看它是否会传递到发送报告作业)
3. 创建了一个新作业(某样东西)，其中我创建了一个新的something.txt工件(查看它是否会传递到发送报告作业)

输出：

Preparing environment
00:01
Running on runner-zqx7qoq-project-4296-concurrent-0 via e3fe672984b4...
Getting source from Git repository
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /<repo>
Checking out 26501c44 as <branch_name>...
Removing something.txt
Skipping Git submodules setup
Downloading artifacts
00:00
Downloading artifacts for something (64950)...
Downloading artifacts from coordinator... ok        id=64950 
responseStatus=200 OK token=zoJwysdq
Executing "step_script" stage of the job script
00:01
$ echo "This is the send-reports before_script"
This is the send-reports before_script
$ echo "in send-reports job"
in send-reports job
$ ls
...<other stuff in repo>
something.txt
Uploading artifacts for successful job
00:01
Uploading artifacts...
WARNING: gl-sast-report.json: no matching files    
ERROR: No files to upload                          
Cleaning up file based variables
00:01
Job succeeded

备注：

• something.txt做了这份工作
• 所有来自上一个作业的工件，而不是进入后续作业。

我只能得出结论，sast模板的内部有些东西不允许工件传播到后续作业。

Answer 1

在第一个作业(sast)中添加以下内容：

  artifacts:
    paths: [gl-sast-report.json]
    reports:
      sast: gl-sast-report.json

在下一个作业(send-reports)中添加以下内容

  artifacts:
    reports:
      sast: gl-sast-report.json

然后，您应该能够在下一个作业(send-reports)中访问报表。

Answer 2

不要将gl-sast-report.json工件引用为sast报告，而是将其引用为常规工件。

所以你要做的就是用这种方式声明工件

artifacts:
  paths:
    - 'gl-sast-report.json'

而不是

reports:
  sast: gl-sast-report.json

Answer 3

我花了整整一天的时间尝试访问内置的IaC扫描仪生成的gl-sast-report.json文件。以下是我最终的成功之处：

首先，不要使用GitLab文档中建议的代码：

include:
  - template: Security/SAST-IaC.latest.gitlab-ci.yml

如果您只想扫描IaC漏洞并在稍后从GitLab UI下载报告，那么上面的代码可以正常工作。但谁想这么做？！我想在下一份工作中访问报告，如果报告中存在medium+漏洞，则失败管道！

如果这是您想要做的，您将需要将官方GitLab IaC扫描器模板中的所有代码添加到管道中，然后进行一些修改。您可以找到这里的最新模板代码，或者使用下面的示例。

修改模板：

# Read more about this feature here: https://docs.gitlab.com/ee/user/application_security/iac_scanning/
#
# Configure SAST with CI/CD variables (https://docs.gitlab.com/ee/ci/variables/index.html).
# List of available variables: https://docs.gitlab.com/ee/user/application_security/iac_scanning/index.html

variables:
  # Setting this variable will affect all Security templates
  # (SAST, Dependency Scanning, ...)
  TEMPLATE_REGISTRY_HOST: 'registry.gitlab.com'
  SECURE_ANALYZERS_PREFIX: "$TEMPLATE_REGISTRY_HOST/security-products"
  SAST_IMAGE_SUFFIX: ""

  SAST_EXCLUDED_PATHS: "spec, test, tests, tmp"

iac-sast:
  stage: test
  artifacts:
    name: sast
    paths:
      - gl-sast-report.json
    #reports:
    #  sast: gl-sast-report.json
    when: always
  rules:
    - when: never
  # `rules` must be overridden explicitly by each child job
  # see https://gitlab.com/gitlab-org/gitlab/-/issues/218444
  variables:
    SEARCH_MAX_DEPTH: 4
  allow_failure: true
  script:
    - /analyzer run

kics-iac-sast:
  extends: iac-sast
  image:
    name: "$SAST_ANALYZER_IMAGE"
  variables:
    SAST_ANALYZER_IMAGE_TAG: 3
    SAST_ANALYZER_IMAGE: "$SECURE_ANALYZERS_PREFIX/kics:$SAST_ANALYZER_IMAGE_TAG$SAST_IMAGE_SUFFIX"
  rules:
    - if: $SAST_DISABLED
      when: never
    - if: $SAST_EXCLUDED_ANALYZERS =~ /kics/
      when: never
    - if: $CI_COMMIT_BRANCH
  

Enforce Compliance:
  stage: Compliance
  before_script:
    - apk add jq
  script:
    - jq -r '.vulnerabilities[] | select(.severity == "Critical") | (.severity, .message, .location, .identifiers[].url)' gl-sast-report.json > results.txt
    - jq -r '.vulnerabilities[] | select(.severity == "High") | (.severity, .message, .location, .identifiers[].url)' gl-sast-report.json >> results.txt
    - jq -r '.vulnerabilities[] | select(.severity == "Medium") | (.severity, .message, .location, .identifiers[].url)' gl-sast-report.json >> results.txt
    - chmod u+x check-sast-results.sh
    - ./check-sast-results.sh

您还需要确保在管道中添加两个阶段(如果您还没有)：

stages:
  # add these to whatever other stages you already have
  - test
  - Compliance

注意:非常重要的是，您的工作试图访问gl- sast -report.json (本例中的“遵从性”)与sast扫描自身(在本例中为“test”)处于同一阶段。如果是，那么您的工作将尝试在报表存在并失败之前访问它。

我将包括管道中引用的shell脚本，以防您也想使用该脚本：

#!/bin/sh

if [ -s results.txt ]; then
        echo ""
        echo ""
        cat results.txt
        echo ""
        echo "ERROR: SAST SCAN FOUND VULNERABILITIES - FIX ALL VULNERABILITIES TO CONTINUE"
        echo ""
        exit 1
fi

这是一个基本脚本，用于检查"results.txt“文件是否包含任何内容。如果存在，则使用代码1退出，以中断管道并打印漏洞。如果文件中没有内容，则脚本退出，代码为0，管道继续(允许部署您的下位文件)。将上面的文件保存为GitLab存储库的根目录中的“check Resul.sh”(与".gitlab-ci.yml“所在的级别相同)。

希望这对外面的人有帮助。