Containerizing Jenkins, DooD and DinD

Learn to containerize Jenkins and run Docker commands with Docker inside Docker (DinD) or Docker outside of Docker (DooD). DinD makes Jenkins' containers invisible to the host, while DooD makes them visible as siblings. Each has its own advantages and drawbacks.

Containerizing Jenkins, DooD and DinD
"Jenkins is the way", https://www.jenkins.io/artwork, Created by CloudBees & abConsulting Network

Note: The sections below use the latest version of the Jenkins image i.e. jenkins/jenkins:latest-jdk17. You should lock this version down.

All source code is available on GitHub.

Basic Configuration

The goal is to run a Jenkins container without the requirement of
creating any additional containers from Jenkins.

docker volume create jenkins-home
docker network create jenkins
docker run \
  --name jenkins \
  --restart unless-stopped \
  --detach \
  --network jenkins \
  --publish mode=ingress,published=80,target=8080 \
  --env JAVA_OPTS="-Djava.awt.headless=true" \
  -v jenkins-home:/var/jenkins_home \
  jenkins/jenkins:latest-jdk17

Run a Jenkins container without any docker related configuration

Alternatively, create a docker-compose.yaml as follows:

version: "3.9"

volumes:
  jenkins-home:

networks:
  jenkins:

services:
  jenkins:
    image: jenkins/jenkins:latest-jdk17
    container_name: jenkins
    networks:
      - jenkins
    environment:
      JAVA_OPTS: "-Djava.awt.headless=true"
    restart: unless-stopped
    ports:
      - "80:8080"
    volumes:
      - jenkins-home:/var/jenkins_home

Docker Compose file to run a Jenkins container.

And start up the container:

#!/usr/bin/env bash

docker compose up -d

Startup Jenkins container using Docker Compose

Executing docker commands from a Jenkins container

Being able to run docker commands is useful for situations when Docker images need to be built or dockerized tests such as those based on test containers need to be executed.

There are two approaches to configure a containerized Jenkins to communicate with docker:

Docker Outside of Docker (DooD)

  • The basic idea is that containers talk to the Docker Daemon/Server process running on the host system.
  • To enable this, the /var/run/docker.sock UNIX socket is mounted against a container to allow the container to communicate with the Docker daemon on the host.
  • According to the docker documentation:

By default, a unix domain socket (or IPC socket) is created at /var/run/docker.sock, requiring either root permission, or docker group membership.

  • Essentially, the Docker daemon listens on the /var/run/docker.sock UNIX socket. This is used by Docker clients to communicate with the server process.

We'll build a custom Jenkins Image with docker installed. Assuming the following directory structure and working directory:

dood/
├── Dockerfile
└── docker-compose.yaml

Docker Outside of Docker (DooD) directory structure for Docker & Docker Compose

We'll first create our custom Dockerfile which adds docker to the base Jenkins image:

FROM jenkins/jenkins:latest-jdk17
USER root
RUN apt-get update -qq \
    && apt-get install -qqy apt-transport-https ca-certificates curl gnupg2 software-properties-common
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
RUN add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/debian \
   $(lsb_release -cs) \
   stable"
RUN apt-get update  -qq \
    && apt-get -y install docker-ce
RUN usermod -aG docker jenkins

Custom Jenkins image with Docker

And then from the dood working directory:

#!/usr/bin/env bash

docker volume create jenkins-home
docker network create jenkins
docker build -t custom/jenkins-with-docker ./

docker run \
  --name jenkins \
  --restart unless-stopped \
  --detach \
  --network jenkins \
  --publish mode=ingress,published=80,target=8080 \
  --env JAVA_OPTS="-Djava.awt.headless=true" \
  -v jenkins-home:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  custom/jenkins-with-docker

Run a Jenkins container which can use Docker via DooD

Alternatively, create a docker-compose.yaml as follows:

version: "3.9"

volumes:
  jenkins-home:

networks:
  jenkins:

services:
  jenkins:
    container_name: jenkins
    build:
      context: ./
      dockerfile: ./Dockerfile
    networks:
      - jenkins
    environment:
      JAVA_OPTS: "-Djava.awt.headless=true"
    restart: unless-stopped
    ports:
      - "80:8080"
    volumes:
      - jenkins-home:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock

Docker Compose file to run a Jenkins container (DooD)

And start up the container:

#!/usr/bin/env bash

docker compose up -d

Startup Jenkins container (DooD) using Docker Compose

Consequences of DooD

  • Any containers started by the Jenkins containers will be its siblings and visible to the host system.

Docker inside of Docker (DinD)

  • The basic idea is that docker itself is started as a container and networked with the Jenkins container.

We'll build a custom Jenkins Image with docker installed. Assuming the following directory structure and working directory:

dind/
├── Dockerfile
└── docker-compose.yaml

Docker Inside of Docker (DinD) directory structure for Docker & Docker Compose

We'll first create our custom Dockerfile which adds docker to the base Jenkins image:

FROM jenkins/jenkins:latest-jdk17
USER root
RUN apt-get update -qq \
    && apt-get install -qqy apt-transport-https ca-certificates curl gnupg2 software-properties-common
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
RUN add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/debian \
   $(lsb_release -cs) \
   stable"
RUN apt-get update  -qq \
    && apt-get -y install docker-ce
RUN usermod -aG docker jenkins

Custom Jenkins image with Docker

And then from the dind working directory:

#!/usr/bin/env bash
docker network create jenkins
docker volume create jenkins-home
docker volume create jenkins-docker-certs
# Start the DinD (Docker in Docker) Container
docker run \
  --name ci-cd-dind \
  --restart unless-stopped \
  --detach \
  --privileged \
  --network jenkins \
  --network-alias docker \
  --publish mode=ingress,published=2376,target=2376 \
  --env DOCKER_TLS_CERTDIR=/certs \
  --volume jenkins-docker-certs:/certs/client \
  --volume jenkins-home:/var/jenkins_home \
  docker:dind
# Start the Jenkins Container
docker build -t custom/jenkins-with-docker ./
docker run \
  --name jenkins \
  --restart unless-stopped \
  --detach \
  --network jenkins \
  --publish mode=ingress,published=80,target=8080 \
  --env JAVA_OPTS="-Djava.awt.headless=true" \
  --env DOCKER_HOST=tcp://docker:2376 \
  --env DOCKER_CERT_PATH=/certs/client \
  --env DOCKER_TLS_VERIFY=1 \
  -v jenkins-home:/var/jenkins_home \
  -v jenkins-docker-certs:/certs/client:ro \
  custom/jenkins-with-docker

Run DinD & Jenkins containers to enable Jenkins to use Docker via DinD

Alternatively, create a docker-compose.yaml as follows:

version: "3.9"

networks:
  jenkins:

volumes:
  jenkins-docker-certs:
  jenkins-home:

services:
  ci-cd-dind:
    image: docker:dind
    container_name: jenkins-docker
    restart: unless-stopped
    privileged: true
    networks:
      jenkins:
        aliases:
          - docker
    ports:
      - "2376:2376"
    environment:
      DOCKER_TLS_CERTDIR: "/certs"
    volumes:
      - jenkins-docker-certs:/certs/client
      - jenkins-home:/var/jenkins_home

  jenkins:
    depends_on:
      - ci-cd-dind
    build:
      context: ./
      dockerfile: ./Dockerfile
    container_name: jenkins
    restart: unless-stopped
    networks:
      - jenkins
    ports:
      - "80:8080"
    environment:
      JAVA_OPTS: "-Djava.awt.headless=true"
      DOCKER_HOST: "tcp://docker:2376"
      DOCKER_CERT_PATH: "/certs/client"
      DOCKER_TLS_VERIFY: 1
    user: root
    volumes:
      - jenkins-home:/var/jenkins_home
      - jenkins-docker-certs:/certs/client

Docker Compose file to run Jenkins & DinD containers.

And start up the container:

#!/usr/bin/env bash

docker compose up -d

Startup Jenkins container (DinD) using Docker Compose

Consequences of DinD

  • Any containers started by the containers are its children and not visible to the host system.
  • The privileged flag is required for DinD. As per the docker documentation:

--privileged flag gives all capabilities to the container, and it also lifts all the limitations enforced by the device cgroup controller. In other words, the container can then do almost everything that the host can do. This flag exists to allow special use-cases, like running Docker within Docker.

In other words, if the container gets compromised, the host gets compromised as well.

Jenkins Configuration and Docker Validation

Regardless of whether DinD or DooD was used, you'll want to install a couple of plugins to use Docker i.e.

After installing the plugins, you can create a pipeline job to verify that everything is working:

```
pipeline {
    agent {
        docker { image 'node:latest' }
    }
    stages {
        stage('Verify Docker Integration') {
            steps {
                sh 'node --version'
            }
        }
    }
}
```

Sample Jenkins pipeline to use for validation.

Verifying DooD Sibling Relationship

Assuming you hadn't already pulled the latest node image, running the docker image ls command on the host will show
you the node image in the image listing.

REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
node         latest    5bb57e984682   42 hours ago   999MB

docker image ls output (DooD)

Verifying DinD Parent/Child Relationship

Assuming you hadn't already pulled the latest node image, running the docker image ls command on the host will not show the latest node image in the listing.

However, if you connect to the jenkins container and check the listing
there it will show up in the image listing pulled from the container:

#!/usr/bin/env bash

docker container exec -it jenkins bash
root@d1ffb64a9278:/# docker image ls
REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
node         latest    5bb57e984682   43 hours ago   999MB

docker image ls output (DinD)

References / Additional Reading

Attributions