Concourse CI/CD (continuous integration/continuous delivery) can create multi-platform Docker images. This blog post describes how.
A multi-platform docker image is one that contains “variants for different architectures”.
Docker images are often created for a single architecture (“instruction set architecture” or “ISA”), typically Intel’s/AMD’s x86-64, but with the advent of ARM64-based offerings such as AWS’s Graviton and Apple’s M1/M2, It’s becoming more common to build multi-platform images to avoid the heavy emulation performance penalty (typically >10x) when running an image on a different architecture. Multi-platform images enable a developer, for example, to run a container just as fast on their Apple M1 laptop as their GCP (Google Cloud Platform) Kubernetes cluster.
Make sure you’re on Concourse 7.9 or later; it has the registry-image resource necessary to create multi-platform images.
Create a Concourse resource, “multi-platform-docker-image”. This is the Docker image you’ll be building. Using the example below, make the following changes:
- replace cunnie/multi-platform with the name of the multi-platform Docker image that you intend to create.
username: cunniewith your Docker username.
((docker_token))with your Docker hub access token
resources: - name: multi-platform-docker-image type: registry-image icon: docker source: repository: cunnie/multi-platform # ← Replace with the name of your Docker image username: cunnie # ← Replace with your Docker Hub username password: ((docker_token)) # ← Replace with your Docker Hub token (or password) tag: latest
Create a job to build the Docker image using the following as an example. Make the following changes:
CONTEXT: deployments/multi-platform-dockerwith the path to your
Dockerfile’s location. In this example, I have a Concourse input resource “deployments”, and it has a subdirectory “multi-platform-docker/”, and that subdirectory has a “Dockerfile”.
IMAGE_PLATFORMdefines the platforms to build. If you’re not sure, use
OUTPUT_OCI: trueis required for multi-platform
image: image/imageis required for OCI output
jobs: - name: build-and-push-multi-platform-docker-image plan: - get: deployments trigger: true - task: build-image-task privileged: true config: platform: linux image_resource: type: registry-image source: repository: concourse/oci-build-task inputs: - name: deployments outputs: - name: image params: CONTEXT: deployments/multi-platform-docker IMAGE_PLATFORM: linux/arm64,linux/amd64 OUTPUT_OCI: true run: path: build - put: multi-platform-docker-image params: image: image/image
Putting it all together, we have the following:
- The Concourse pipeline
- The Concourse pipeline definition (YAML)
- The Dockerfile
- The generated Docker image
The resulting Docker image is simple: it prints out the architecture of the underlying kernel (i.e. “aarch64” for ARM, “x86_64” for Intel) and then exits. See for yourself:
docker run -it --rm cunnie/multi-platform
Update your Concourse pipelines to use the new resource
instead of the old, deprecated
docker-image, otherwise your pipelines will
pull the old (pre-multi-platform) image, and you’ll be sad.
jobs: name: unit plan: - get: sslip.io trigger: true - config: image_resource: source: repository: cunnie/fedora-golang-bosh - type: docker-image + type: registry-image
If your Docker image needs to be built slightly differently for different
platforms, use the
environment variable. In the following
snippet, we use the
TARGETARCH to download and install tha Golang-compiled
executables for the appropriate architecture:
ARG TARGETARCH # amd64, arm64 (so I can run on AWS graviton2) RUN curl -L https://github.com/cunnie/sslip.io/releases/download/2.6.1/sslip.io-dns-server-linux-$TARGETARCH \ -o /usr/sbin/sslip.io-dns-server; \ chmod 755 /usr/sbin/sslip.io-dns-server
Corrections & Updates
Gotcha: update your pipelines to use the new
registry-image instead of the
docker-image, lest your pipelines don’t pull the new image.
I updated the post to require Concourse 7.9, which allowed me to remove the custom Concourse resource type.