Jenkins Pipeline and semaphores

jenkins pipeline pause between stages
jenkins pipeline synchronize
jenkins pipeline wait for stage to complete
jenkins lockable resources
jenkins pipeline steps
jenkins pipeline break stage
jenkins manual stage
jenkins pipeline sh

I'm building a Jenkins job that will run all my staging tests continuously, but not all at once (they rely on shared hardware). So, I'm creating parallel jobs, with a semaphore to ensure that only a limited amount are run at once. Here's a simplified version of my pipeline that reproduces the issue:

import java.util.concurrent.Semaphore

def run(job) {
  return {
    this.limiter.acquire();
    try {
      println "running ${job}"
      build job
      println "finished ${job}"
    } finally {
      this.limiter.release();
    }
  }
}

def getJobs() {
  def allJobs = Jenkins.getInstance().getJobNames()
  def stagingJobs = []
  for(String job : allJobs) {
    if (job.startsWith("staging/temp")) {
      stagingJobs.add(job)
    }
  }
  println "${stagingJobs.size()} jobs were found."
  return stagingJobs
}

this.limiter = new Semaphore(2)
def jobs = [:]
for (job in getJobs()) {
  jobs[job] = run(job)
}
parallel jobs

When I run without the semaphores, everything works fine. But with the code above, I get nothing outputted except:

[Pipeline] echo
6 jobs were found.
[Pipeline] parallel
[Pipeline] [staging/temp1] { (Branch: staging/temp1)
[Pipeline] [staging/temp2] { (Branch: staging/temp2)
[Pipeline] [staging/temp3] { (Branch: staging/temp3)
[Pipeline] [staging/temp4] { (Branch: staging/temp4)
[Pipeline] [staging/temp5] { (Branch: staging/temp5)
[Pipeline] [staging/temp6] { (Branch: staging/temp6)

If I view the pipeline steps, I can see the first two jobs start, and their log messages output. However, it seems like the runner never receives a notification that the staging jobs finish. As a result, the semaphore never releases and the other 4 jobs never manage to start. Here's a thread dump mid test, after the downstream builds have definitely finished:

Thread #7
    at DSL.build(unsure what happened to downstream build)
    at WorkflowScript.run(WorkflowScript:9)
    at DSL.parallel(Native Method)
    at WorkflowScript.run(WorkflowScript:38)
Thread #8
    at DSL.build(unsure what happened to downstream build)
    at WorkflowScript.run(WorkflowScript:9)
Thread #11
    at WorkflowScript.run(WorkflowScript:6)
Thread #12
    at WorkflowScript.run(WorkflowScript:6)

Eventually it times out with several java.lang.InterruptedException errors.

Is it possible to use semaphores in a pipeline, or is there a better way to ensure only a portion of jobs run at once? I would rather avoid spinning up nodes for what amounts to a simple test runner.


Since at least a year there's a plugin which will help you to get what you intent and there's also the option to use this plugin in pipeline jobs, the plugin is Lockable Plugin Resource

Basically you wrap your share resource and the job will queue before the lock statement if the resource is not free.

If you are interested in parallelizing your tests you also can have a look at the Parallel Test Executor Plugin

Concurrent Step, Jenkins and Semaphore belong to "Continuous Integration" category of the tech me the advice which one is best to use for my case Azure pipeline or jenkins. Developers describe Jenkins as " An extendable open source continuous integration server ". In a nutshell Jenkins CI is the leading open-source continuous integration server. Built with Java, it provides over 300 plugins to support building and testing virtually any project. On the other hand, Semaphore is detailed as " The continuous integration and delivery platform powering the world’s best engineering teams ".


Jenkins vs Semaphore, So, I'm creating parallel jobs, with a semaphore to ensure that only a limited amount are run at once. Here's a simplified version of my pipeline  When comparing Semaphore CI vs Jenkins, the Slant community recommends Jenkins for most people. In the question “What are the best continuous integration tools?” Jenkins is ranked 3rd while Semaphore CI is ranked 10th. The most important reason people chose Jenkins is:


The Concurrent Step plugin was just released and should work nicely for this use case.

Wtih this, you can simplify your code:

def semaphore = createSemaphore permit:2

def run(job) {
  return {
    acquireSemaphore (semaphore) {
      println "running ${job}"
      build job
      println "finished ${job}"
    }
  }
}

...

[JENKINS-44545] Pipeline fails when using java.util.concurrent , Now we'll see how we can define our own pipelines on Semaphore. Let's say that our CI build consists of 4 steps: Compile; Run unit tests; Run integration tests  I'm building a Jenkins job that will run all my staging tests continuously, but not all at once (they rely on shared hardware). So, I'm creating parallel jobs, with a semaphore to ensure that only a limited amount are run at once. Here's a simplified version of my pipeline that reproduces the issue:


Customizing your pipeline, Jenkins processes the jenkinsfile on the master, but I believe as gitlab-runner pulls it a subsequent commit could get processed on a completely different runner. Jenkins - An extendable open source continuous integration server. Semaphore - The continuous integration and delivery platform powering the world’s best engineering teams. Travis CI - A hosted continuous integration service for open source and private projects


Limit pipeline concurrency using named semaphores (#20481 , Jenkins plugin to synchronize status among parallel stages in pipeline. def semaphore = createSemaphore permit:1 def out2=0 parallel( semaphore1:  The following plugin provides functionality available through Pipeline-compatible steps. Read more about how to integrate steps into your Pipeline in the Steps section of the Pipeline Syntax page. For a list of other such plugins, see the Pipeline Steps Reference page.


jenkinsci/concurrent-step-plugin: Jenkins plugin to use utils , Jenkins Job Locks adhere to the logic of a mutex. A mutex is an OS Control Construct where certain threads are blocked from execution when it  Jenkins Pipeline (or simply "Pipeline" with a capital "P") is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins. A continuous delivery (CD) pipeline is an automated expression of your process for getting software from version control right through to your users and customers.