K8s plugin for Jenkins: Always running two separate containers inside a pod

jenkins kubernetes plugin examples
jenkins kubernetes pod template example
jenkins kubernetes declarative pipeline
jenkins pipeline agent ( kubernetes)
jenkins kubernetes pipeline example
jenkins kubernetes workspace-volume
jenkins kubernetes plugin declarative pipeline
jenkins kubernetes deployment yaml

I have created a Dockerfile (for a Node JNLP slave which can be used with the Kubernetes Plugin of Jenkins ). I am extending from from the official image jenkinsci/jnlp-slave

FROM jenkinsci/jnlp-slave

USER root


MAINTAINER Aryak Sengupta <aryak.sengupta@hyland.com>
LABEL Description="Image for NodeJS slave"

COPY cert.crt /usr/local/share/ca-certificates
RUN update-ca-certificates

RUN curl -sL https://deb.nodesource.com/setup_8.x | bash \
    && apt-get install -y nodejs

ENTRYPOINT ["jenkins-slave"]

I have this image saved inside my Pod template (in K8s plugin configuration). Now, when I'm trying to run a build on this slave, I find that two containers are getting spawned up inside the Pod (A screenshot to prove the same.).

My Pod template looks like this:

And my Kubernetes configuration looks like this:

Now if I do a simple docker ps, I find that there are two containers which started up (Why?):

Now, inside the Jenkins Job configuration of Jenkins, whatever I add in the build step, the steps get executed in the first container .

Even if I use the official Node container inside my PodTemplate, the result is still the same:

I have tried to print the Node version inside my Jenkins Job, and the output is "Node not found" . Also, to verify my haunch, I have done a docker exec into my second container and tried to print the Node version. In this case, it works absolutely fine.

This is what my build step looks like:

So, to boil it down, I have two major questions:

  1. Why does two separate (one for JNLP and one with all custom changes) containers start up whenever I fire up the Jenkins Job?
  2. Why is my job running on the first container where Node isn't installed? How do I achieve the desired behaviour of building my project with Node using this configuration?

What am I missing?

P.S. - Please do let me know if the question turns out to be unclear in some parts.

Edit: I understand that this can be done using the Pipeline Jenkins plugin where I can explicitly mention the container name, but I need to do this from the Jenkins UI. Is there any way to specify the container name along with the slave name which I am already doing like this:

The Jenkins kubernetes plugin will always create a JNLP slave container inside the pod that is created to perform the build. The podTemplate is where you define the other containers you need in order to perform your build.

In this case it seems you would want to add a Node container to your podTemplate. In your build you would then have the build happen inside the named Node container.

You shouldn't really care where the Pod runs. All you need to do is make sure you add a container that has the resources you need (like Node in this case). You can add as many containers as you want to a podTemplate. I have some with 10 or more containers for steps like PMD, Maven, curl, etc.

I use a Jenkinsfile with pipelines.

podTemplate(cloud: 'k8s-houston', label: 'api-hire-build', 
  containers: [
    containerTemplate(name: 'maven', image: 'maven:3-jdk-8-alpine', ttyEnabled: true, command: 'cat'),
    containerTemplate(name: 'pmd', image: 'stash.company.com:8443/pmd:pmd-bin-5.5.4', alwaysPullImage: false, ttyEnabled: true, command: 'cat')
  ],
  volumes: [
    persistentVolumeClaim(claimName: 'jenkins-pv-claim', mountPath: '/mvn/.m2nrepo')
  ]
)
{
  node('api-hire-build') {
    stage('Maven compile') {
      container('maven') {
        sh "mvn -Dmaven.repo.local=/mvn/.m2nrepo/repository clean compile"
      }
    }
    stage('PMD SCA (docker)') {
      container('pmd') {
        sh 'run.sh pmd -d "$PWD"/src -f xml -reportfile "$PWD"/target/pmd.xml -failOnViolation false -rulesets java-basic,java-design,java-unusedcode -language java'
        sh 'run.sh pmd -d "$PWD"/src -f html -reportfile "$PWD"/target/pmdreport.html -failOnViolation false -rulesets java-basic,java-design,java-unusedcode -language java'
        sh 'run.sh cpd --files "$PWD"/src --minimum-tokens 100 --failOnViolation false --language java --format xml > "$PWD"/target/duplicate-code.xml'
      }
      archive 'target/duplicate-code.xml'
      step([$class: 'PmdPublisher', pattern: 'target/pmd.xml'])
    }
  }
}

jenkinsci/kubernetes-plugin: Jenkins plugin to run dynamic , Jenkins plugin to run dynamic agents in a Kubernetes/Docker environment Multiple containers can be defined for the agent pod, with shared resources, like or using yamlFile to keep the pod template in a separate KubernetesPod.yaml file 2 K8s plugin for Jenkins: Always running two separate containers inside a pod May 31 '18 2 Vega: Use filter as input and dynamically change data URL Sep 23 '19 2 Installing Django with pip May 4 '14

Alright so I've figured out the solution. mhang li's answer was the clue but he didn't explain it one bit.

Basically, you need to modify the official Jenkins Slave image found here and modify it to include the changes for your slave as well. Essentially, you are clubbing the JNLP and Slave containers into one and building a combined image.

The modification format will just look like this (picking up from the Dockerfile linked)

FROM jenkins/slave:3.27-1
MAINTAINER Oleg Nenashev <o.v.nenashev@gmail.com>
LABEL Description="This is a base image, which allows connecting Jenkins agents via JNLP protocols" Vendor="Jenkins project" Version="3.27"

COPY jenkins-slave /usr/local/bin/jenkins-slave

**INCLUDE CODE FOR YOUR SLAVE. Eg install node, java, whatever**

ENTRYPOINT ["jenkins-slave"] # Make sure you include this file as well

Now, name the slave container jnlp (Reason - bug). So now, you will have one container that spawns which will be your JNLP + Slave. All in all, your Kubernetes Plugin Pod Template will look something like this. Notice the custom url to the docker image I have put in. Also, make sure you don't include a Command To Run unless you need one.

Done! Your builds should now run within this container and should function exactly like you programmed the Dockerfile!

Kubernetes plugin, For example, launching agents in containers using the Docker Plugin or using managed my Docker host(s) for running Jenkins agents several different ways: It will always contain the default JNLP container that runs the Jenkins agent jar There are at least two ways to configure pod templates – in the  Sometimes it is neccessary to check on a hung test. Each pod is running VNC. To check on one of the browser nodes via VNC, it's reccomended that you proxy, since we don't want to expose a service for every pod, and the containers have a weak VNC password. Replace POD_NAME with the name of the pod you want to connect to.

To set Container Template -> Name as jnlp. https://issues.jenkins-ci.org/browse/JENKINS-40847

Jenkins and Kubernetes, Kubernetes, the container orchestration platform is rapidly becoming popular. Kubernetes is able to run your Jenkins workloads as long as they are run in container. You can also specify multiple scopes of quota per one namespace. by the Kubernetes plugin when you are specifying a Pod Template. Services - The building block of Kubernetes Load Balancing. Once you have your application running in Kubernetes, its’ scheduler makes sure that you have the number of desired pods always running. This means that application pods are created and deleted unexpectedly and you should not rely on a particular pod.

Effectively using Kubernetes plugin with Jenkins, It's very flexible to use all kinds of mechanisms provided by Kubernetes, including multiple containers in the Pod, sharing the network among them, sharing the  A working Kubernetes deployment is called a cluster. You can visualize a Kubernetes cluster as two parts: the control plane, which consists of the master node or nodes, and the compute machines, or worker nodes. Worker nodes run pods, which are made up of containers.

Dynamic Jenkins Agent from Kubernetes, Use open source containers to configure, setup and implement a mean the ability to deploy, scale, and operate with multiple containers from one place. In our case we use the 'RUN' command to install different common Jenkins plugins, To interact with a pod inside the Kubernetes cluster you need to  InfoQ Homepage Articles Scaling Docker with Kubernetes. to ensure that it is always running. If a container is killed, it will try to start a new one. example we could have a pod with two

How to Setup Scalable Jenkins on Top of a Kubernetes Cluster , Set Up a CI/CD Pipeline with a Jenkins Pod in Kubernetes (Part 2). By reboot your computer and keep the number of running apps to a minimum. 2. Once again we'll need to set up the Socat Registry proxy container to push One of the plugins being installed is Kubernetes Continuous Deploy, which  Defines the Bash script that starts the Jenkins Agent within a Build Agent Container. StatefulSet. cjoc. Defines a Pod for the cjoc Container, allocates a persistent volume for its JENKINS_HOME directory and ensures that one such Pod is always running. Service. cjoc. Defines a Service front-end for the cjoc Pod and assigns TCP ports 80 and

Comments
  • What is your podTemplate?
  • @DanielWatrous I have updated the question with the Pod template. Thanks
  • I have also updated the question with the Jenkins Job output. Thought, I should let you know.
  • Daniel, I have exactly the same configuration. The only difference is I am doing this without a Jenkinsfile. I'm going for a manual configuration for the Job. Over there I am mentioning the same label which I've mentioned for the Pod template.
  • Now I am using the official Node container in the PodTemplate, still I am getting the same result, which is, Node: Not found error.
  • My hunch is that your Execute shell command has to look different in order to use the node container. I think that shell command is running on the default JNLP container.
  • I just tried to create a manual job, but I don't see any way in Execute Shell to identify the desired container in which to run the command. Is there some reason not to use a pipeline script?
  • Ho my god, ho my god... I spent my day looking at tutorial and that is the first real configuration I found... Thanks a lot...