在Jenkins声明性管道中运行在同一代理上的并行阶段

Question

我一直在使用Jenkins管道进行类似的并行测试：

pipeline {
    agent { label 'php' }
    stages {
        stage('build') {
            steps {
                ...
            }
        }
        stage('test') {
            parallel {
                stage('unit tests') {
                    steps {
                        ...
                    }
                }
                stage('integration tests') {
                    steps {
                        ...
                    }
                }
            }
        }
        stage('deploy') {
            steps {
                ...
            }
        }
    }
}

管道在单个代理上运行良好，但是现在我想添加一个带有手动输入的步骤，该步骤不应该保持代理运行。代理是云服务器，按需启动并停止。

为此，我将“无”代理分配给管道，将实际代理分配到所有阶段，但请求输入的阶段除外：

pipeline {
    agent none
    stages {
        stage('build') {
            agent { label 'php' }
            steps {
                ...
            }
        }
        stage('test') {
            parallel {
                stage('unit tests') {
                    agent { label 'php' }
                    steps {
                        ...
                    }
                }
                stage('integration tests') {
                    agent { label 'php' }
                    steps {
                        ...
                    }
                }
            }
        }
        stage('deploy staging') {
            agent { label 'php' }
            steps {
                ...
            }
        }
        stage('confirm release') {
            steps {
                input { message 'release?' }
            }
        }
        stage('deploy production') {
            agent { label 'php' }
            steps {
                ...
            }
        }
    }
}

在接下来的阶段中所需的工件将使用stash和unstash进行传递。这是可行的，但有两个麻烦：

1. 每个阶段之间的存储和解藏都会导致开销。我希望有所有的阶段，直到手动输入在一个工作空间中执行。但是agent只允许在包含步骤的阶段中使用。
2. 每个并行阶段阻塞一个代理。因此，与以前不同，如果只有一个自由代理，就没有并行执行。

如果不可能的话，我可以接受(1)，但是在一个代理上并行执行对我来说很重要。在我目前的设计中有可能吗？如果没有，我有什么选择来实现类似的目标？

Answer 1

我修改了您的Jenkins文件，如下所示。

pipeline {
agent { label 'agent01'}
stages {
    stage('build') {
        steps {
            echo "build"
            sh "ls -lrt && touch build.txt"
        }
    }
    stage('test') {
        parallel {
            stage('unit tests') {
                steps {
                    echo "unit tests"
                    sh "ls -lrt && touch ut.txt"
                    sleep(time:10,unit:"SECONDS")
                }
            }
            stage('integration tests') {
                steps {
                    echo "integration tests"
                    sh "ls -lrt && touch it.txt"
                    sleep(time:10,unit:"SECONDS")
                }
            }
        }
    }
    stage('deploy staging') {
        agent { label 'master' }
        steps {
            echo "deploy staging"
            sh "ls -lrt && touch dep.txt"
        }
    }
    stage('confirm release') {
        agent { label 'master' }
        steps {
            echo "RELEASE?"
            sh "ls -lrt && touch release.txt"
        }
    }
    stage('deploy production') {
        steps {
            echo "deploy production"
            sh "ls -lrt && touch produ.txt"
        }
    }
}
}

1. 您不需要存储和取消存储文件，因为它运行在相同的节点和工作区上。
2. 在所有阶段使用通用节点，需要用户输入的节点除外。

输出是

    Running in Durability level: MAX_SURVIVABILITY
[Pipeline] node
Running on agent01 in /var/jenkins_home/workspace/sandbox-pipeline
[Pipeline] {
[Pipeline] stage
[Pipeline] { (build)
[Pipeline] echo
build
[Pipeline] sh
[sandbox-pipeline] Running shell script
+ ls -lrt
total 0
+ touch build.txt
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (test)
[Pipeline] parallel
[Pipeline]     [unit tests] { (Branch: unit tests)
[Pipeline]     [integration tests] { (Branch: integration tests)
[Pipeline]     [unit tests] stage
[Pipeline]     [unit tests] { (unit tests)
[Pipeline]     [integration tests] stage
[Pipeline]     [integration tests] { (integration tests)
[Pipeline]     [unit tests] echo
[unit tests] unit tests
[Pipeline]     [unit tests] sh
[unit tests]     [sandbox-pipeline] Running shell script
[Pipeline]     [integration tests] echo
[integration tests] integration tests
[Pipeline]     [integration tests] sh
[unit tests] + ls -lrt
[unit tests] total 0
[unit tests] -rw-r--r-- 1 jenkins docker 0 Oct 19 13:53 build.txt
[unit tests] + touch ut.txt
[integration tests]     [sandbox-pipeline] Running shell script
[Pipeline]     [unit tests] sleep
[integration tests] + ls -lrt
[integration tests] total 0
[integration tests] -rw-r--r-- 1 jenkins docker 0 Oct 19 13:53 build.txt
[integration tests] -rw-r--r-- 1 jenkins docker 0 Oct 19 13:53 ut.txt
[integration tests] + touch it.txt
[unit tests] Sleeping for 10 sec
[Pipeline]     [integration tests] sleep
[integration tests] Sleeping for 10 sec
[Pipeline]     [unit tests] }
[Pipeline]     [unit tests] // stage
[Pipeline]     [unit tests] }
[Pipeline]     [integration tests] }
[Pipeline]     [integration tests] // stage
[Pipeline]     [integration tests] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (deploy staging)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/sandbox-pipeline
[Pipeline] {
[Pipeline] echo
deploy staging
[Pipeline] sh
[sandbox-pipeline] Running shell script
+ ls -lrt
total 0
+ touch dep.txt
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (confirm release)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/sandbox-pipeline
[Pipeline] {
[Pipeline] echo
RELEASE?
[Pipeline] sh
[sandbox-pipeline] Running shell script
+ ls -lrt
total 0
-rw-r--r-- 1 jenkins 115 0 Oct 19 13:58 dep.txt
+ touch release.txt
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (deploy production)
[Pipeline] echo
deploy production
[Pipeline] sh
[sandbox-pipeline] Running shell script
+ ls -lrt
total 0
-rw-r--r-- 1 jenkins docker 0 Oct 19 13:53 build.txt
-rw-r--r-- 1 jenkins docker 0 Oct 19 13:53 ut.txt
-rw-r--r-- 1 jenkins docker 0 Oct 19 13:53 it.txt
+ touch produ.txt
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS