How to push multiple images needed for docker-compose to GitLab registry in GitLab CI?

gitlab-runner docker-compose
.gitlab-ci.yml docker example
gitlab docker-compose
ci_registry_image
gitlab docker executor
gitlab docker registry pricing
docker:dind
gitlab ci multiple images

I recently got into CI/CD, and a good starting point for me was GitLab, since they provide an easy interface for that and i got started about what pipelines and stages are, but i have run into some kind of contradictory thought about GitLab CI running on Docker.

My app runs on Docker Compose. It contains (blah blah) that makes it easy to build & run containers. Each service in the Docker Compose creates a single Docker container, excepting the php-fpm one, which is able to do the thing called "horizontal scale", so I can scale it later.

I will use that Docker Compose for production, I am currently using it in development and I want to use it too in CI/CD pipelines.

However the .gitlab-ci.yml provides support for only one image, so I have to build it and push it to either their GitLab Registry or Docker Hub in order to pull it later in the CI/CD process.

How can I build my Docker Compose's service as a single image in order to push it to the Registry/Docker so I can pull it in the CI/CD?

My project contains a docker folder and a docker-compose.yml. In the docker folder, each service has its own separate directory (php-fpm, nginx, mysql, etc.) and each one (prepare yourself) contains a Dockerfile with build details, especially the php-fpm one (deps and libs are strong with this one)

Each service in the docker-compose.yml has a build context in each of their own folder.

If I was unclear, I can provide additonal info.

You don't need dind to run a docker-compose stack. You can run multiple docker-compose up commands.

acceptance_testing:
  stage: test
  before_script:
    - docker-compose -p $CI_JOB_ID up -d
  script:
    - docker-compose -p $CI_JOB_ID exec -T /run/your/test/suite.sh
  after_script:
    - docker-compose -p $CI_JOB_ID down -v --remove-orphans || true

Building Docker images with GitLab CI/CD, You can add as many services you need to .gitlab-ci.yml or manually modify config.toml . Any image found at Docker Hub or your private Container Registry can  Push multiple images with a single docker-compose push command. Vanilla Docker in GitLab CI/CD. Before going into how to setup Docker Compose in a pipeline, let's take a quick look at how to use plain Docker-CLI in GitLab CI/CD. To be able to build Docker images, we need the Docker CLI and a Docker Service/Dameon. Therefore, to build in a

Using Docker images, Description Developers create multiple containers based on the same It seems that I can't push a docker image to the new registry when I specify the project as such - a typical scenario when using docker-compose . Those images are then pulled by Gitlab CI to run tests associated with each project. SysAdmin Automatically build and push Docker images using GitLab CI. How to setup and proper maintainable workflow for managing your own Docker images: automate everything from your commit to the hub.

However the .gitlab-ci.yml provides support for only one image

This is not true. From the official documentation:

Your image will be named after the following scheme:

<registry URL>/<namespace>/<project>/<image>

GitLab supports up to three levels of image repository names. Following examples of image tags are valid:

registry.example.com/group/project:some-tag
registry.example.com/group/project/image:latest
registry.example.com/group/project/my/image:rc1

So the solution to your problem is simple - just build individual images and push them to GitLab container registry under different image name. The following config should give you an idea:

.template: &build_template
  image: docker:stable
  services:
    - docker:dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker pull $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest || true
    - if [ -z ${CI_COMMIT_TAG+x} ];
      then docker build
        --cache-from $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest
        --file $DOCKERFILE_NAME
        --tag $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_SHA
        --tag $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_TAG
        --tag $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest . ;
      else docker build
        --cache-from $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest
        --file $DOCKERFILE_NAME
        --tag $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_SHA
        --tag $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest . ;
      fi
    - docker push $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_SHA
    - if [ -z ${CI_COMMIT_TAG+x} ]; then
        docker push $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_TAG;
      fi
    - docker push $CI_REGISTRY_IMAGE/$IMAGE_NAME:latest

build:image1:
  <<: *build_template
  variables:
    IMAGE_NAME: image1
    DOCKERFILE_NAME: Dockerfile.1

build:image2:
  <<: *build_template
  variables:
    IMAGE_NAME: image2
    DOCKERFILE_NAME: Dockerfile.2

And you should be able to pull the same image using $CI_REGISTRY_IMAGE/$IMAGE_NAME:$CI_COMMIT_SHA in later pipeline jobs or your compose file (provided that the variables are passed to where you run your compose file).

Support specifying a multi-level image name in Gitlab Container , gitlab-ci.yml provides support for only one image, so I have to build it and push it to either their GitLab Registry or Docker Hub in order to pull it  You can specify a tagged image to be used as a cache source for the docker build command by using the --cache-from argument. Multiple images can be specified as a cache source by using multiple --cache-from arguments. Keep in mind that any image that’s used with the --cache-from argument must first be pulled (using docker pull) before it can

How to push multiple images needed for docker-compose to GitLab , Understand how you can build Docker images using GitLab CI/CD; Build a to the GitLab Container Registry; Build and push multiple images with Docker Push multiple images with a single docker-compose push command. the docker build command you need to issue locally and in the pipeline. Building Docker images with GitLab CI/CD GitLab CI/CD allows you to use Docker Engine to build and test docker-based projects. TIP: Tip: This also allows to you to use docker-compose and other docker-enabled tools.

Building Docker Images using Docker Compose and Gitlab CI/CD, To store our source code, run CI/CD tasks, and host the Docker registry, we need a GitLab instance installed on an Ubuntu 16.04 server. GitLab  GitLab Community Edition is a self-hosted software suite that provides Git repository hosting, project tracking, CI/CD services, and a Docker image registry, among other features. In this tutorial we will use GitLab’s continuous integration service to build Docker images from an example Node.js app. These images will then be tested and

How To Build and Host Docker Images with GitLab, Automatically build and push Docker images using GitLab CI. How to One runner for multiple runners 🤔 docker-compose run --rm gitlab-runner register -n \ --url https://gitlab.com/ That's why I recommend removing the image after pushing it, in order to keep your local registry clean and saving storage. Then we push it to our Gitlab Registry. It has all the steps in it required to build the image, from setting the base image to building the project. {location inside gitlab-ci} {new

Comments
  • This is a pretty common setup, nothing unusual.
  • I have pulled a Docker image and ran the docker-compose commands to run my containers.