If you have ever wondered how to build multi-arch containers to run on ppc64le, x86, ARM , and/or s390x with GitHub Actions, then this article is for you.
GitHub Actions is a continuous integration and continuous development (CI/CD) tool on GitHub that allows users to automate the build, test, and deployment of code. It officially supports building on x86 and ARM CPU architectures via self-hosted runners.
This tutorial is for users who want to build multiple architecture (multi-arch) container images on GitHub Actions. To achieve this, we will cross-compile using Docker Buildx.
Before you begin
Let’s briefly discuss multi-arch images and the Docker Buildx tool.
What is a multiple architecture (multi-arch) image?
Multi-arch container images follow the Build Once, Deploy Anywhere principle. That is, multi-arch images are built to run on multiple CPU platforms without the need to explicitly specify which platform it's running on. The container engine automatically pulls the correct image at runtime based on the host CPU architecture.
The following links provide more information about multi-arch images.
To build a multi-arch container image, you need access to native hardware. But what if you don’t have access or if adding new hardware to the existing CI setup is too complex? In such cases, you can use container build tools like Buildx, Ko, Buildah, and so on, which emulate the target CPU architecture for building images. There are a few limitations and drawbacks to this approach that are discussed at the end of this tutorial.
What is Buildx?
Buildx is a build plugin from Docker that uses QEMU emulation to cross-build container images. QEMU emulates all instructions of a target CPU instruction set on the host processor where it runs.
NOTE: Buildx is supported in Docker from version 19.03 onwards and requires at least Linux kernel version 4.8 for QEMU emulation to perform cross builds.
Now, let's take a quick look at how cross-compilation works by building an IBM Power (ppc64le) image on an x86 host using Docker Buildx.
Prerequisites
Following are the prerequisites for this tutorial:
- Access to x86 host with Docker installed to build images (recommended operating systems are Ubuntu, Fedora, CentOS, or Debian).
- Access to an IBM Power host to test the image and prove that cross-compilation is possible.
Estimated time
This tutorial creates a simple Docker image that might take around 15-20 minutes to perform. However, for more complex projects, the time to create a Docker image may be longer. For example, a less complex project might take 5 mins, a moderately complex may take 15-20 minutes, and a very complex project might take 30 minutes or more.
Cross-compile and build ppc64le image on x86 host using Docker Buildx
Following are the steps to cross-compile and build an IBM Power image on a local x86 host using Docker Buildx. These steps demonstrate how cross-compilation works and are not required to build a multi-arch image using GitHub Actions.
Step 1: Set up QEMU static binaries
The following command sets up the QEMU static binaries. QEMU requires static binaries to emulate the target architectures.
root@x86:~# docker run --rm --privileged tonistiigi/binfmt:latest --install all
installing: arm64 OK
installing: riscv64 OK
installing: mips64le OK
installing: mips64 OK
installing: arm OK
installing: s390x OK
installing: ppc64le OK
{
"supported": [
"linux/amd64",
"linux/arm64",
"linux/riscv64",
"linux/ppc64le",
"linux/s390x",
"linux/386",
"linux/mips64le",
"linux/mips64",
"linux/arm/v7",
"linux/arm/v6"
],
"emulators": [
"qemu-aarch64",
"qemu-arm",
"qemu-mips64",
"qemu-mips64el",
"qemu-ppc64le",
"qemu-riscv64",
"qemu-s390x"
]
}
Step 2: Set up Buildx builder
Builder is an isolated environment for building images. It requires a driver which is used as a backend to build images. There are multiple drivers supported such as, Kubernetes, Docker, remote, and so on. You can find a list of all supported drivers here.
In the following code sample:
- --driver docker-container is used because it builds multi-arch images.
- --platform linux/amd64, linux/ppc64le aids Buildx in setting up the builder for building x86 and IBM Power images.
- --driver-opt "network=host" --buildkitd-flags "--allow-insecure-entitlement network.host --allow-insecure-entitlement security.insecure" are optional flags.
You can find more information on Buildx builder options in the docker buildx_create documentation.
root@x86:~# docker buildx create --name multiarch --platform linux/amd64,linux/ppc64le --driver docker-container --driver-opt "network=host" --buildkitd-flags "--allow-insecure-entitlement network.host --allow-insecure-entitlement security.insecure" --use
multiarch
root@x86:~# docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS
multiarch * docker-container
multiarch0 unix:///var/run/docker.sock inactive linux/amd64*, linux/ppc64le*
default docker
default default running linux/amd64, linux/386, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/arm/v7, linux/arm/v6
Step 3: Cross-compile ppc64le image on x86 host
In the following code, the --platform linux/ppc64le flag builds the image for the ppc64le CPU architecture.
By default, Buildx keeps the new image in an isolated environment, hence you must explicitly specify Buildx to either load the image into the local Docker daemon using –-load or push it into an image registry using –-push (on x86 host). Refer to the Docker Buildx build documentation for more information.
Note: You can run these steps on any operating system. You can also replace Buildx with BuildAH. Refer to Building multi-arch container images with GitHub Actions and Buildah.
root@x86:~# docker buildx build --load --platform linux/ppc64le --tag quay.io/siddhesh_ghadi/multi-arch-gha:latest --builder multiarch -f Dockerfile .
[+] Building 9.4s (7/7) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 186B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for registry.access.redhat.com/ubi8/ubi-minimal:latest 1.6s
=> [1/2] FROM registry.access.redhat.com/ubi8/ubi-minimal:latest@sha256:6e79406e33049907e875cb65a31ee2f0575f47afa0f06e3a2a9316b01ee3 3.2s
=> => resolve registry.access.redhat.com/ubi8/ubi-minimal:latest@sha256:6e79406e33049907e875cb65a31ee2f0575f47afa0f06e3a2a9316b01ee3 0.0s
=> => sha256:c7a5b2f22050c29571ca75bd0770089e48f198610dc27abd217eb457407937fd 1.74kB / 1.74kB 0.3s
=> => sha256:77d8a8d64c347832bb6270961f851fb64072b02218d99c93f6c63b3e100b59c7 41.18MB / 41.18MB 2.0s
=> => extracting sha256:77d8a8d64c347832bb6270961f851fb64072b02218d99c93f6c63b3e100b59c7 1.1s
=> => extracting sha256:c7a5b2f22050c29571ca75bd0770089e48f198610dc27abd217eb457407937fd 0.0s
=> [2/2] RUN echo "Container for $(uname -m) CPU architecture." > /opt/output 1.8s
=> exporting to oci image format 2.7s
=> => exporting layers 0.1s
=> => exporting manifest sha256:b7f4563f269a62d3a1c0d8c2bb44c4dfa943680ea02a1d3b0dc026880d86a456 0.0s
=> => exporting config sha256:49ab08f4c0263b34b2ce4042c59280cbeb4386dfa08ff7fa5637c025c6f4d1da 0.0s
=> => sending tarball 2.6s
=> importing to docker
Step 4: Verify built image
As shown in the following command, Docker Buildx uses QEMU to emulate the ppc64le CPU instructions on an x86 host virtual machine (VM) to create an IBM Power container image.
root@x86:~# docker images quay.io/siddhesh_ghadi/multi-arch-gha:latest
REPOSITORY TAG IMAGE ID CREATED SIZE
quay.io/siddhesh_ghadi/multi-arch-gha latest 49ab08f4c026 8 minutes ago 119MB
root@x86:~# docker inspect quay.io/siddhesh_ghadi/multi-arch-gha:latest |grep Arch
"Architecture": "ppc64le",
Build multi-arch image with GitHub Actions
In the first section, you cross-compiled and built an IBM Power image on a local x86 host using Docker Buildx. In this section, you will perform similar steps with the help of GitHub Actions workflow.
You can use a simple Dockerfile to demonstrate multi-arch image build for x86 and IBM Power CPU architecture on GitHub Actions. Before you set up a GitHub Actions workflow for multi-arch image, ensure that the Dockerfile builds without errors on the target platforms, which in this example, are x86 and IBM Power. Build an image on native hardware or perform a cross-build locally as shown in previous steps.
NOTE: To create a multi-arch image, you must ensure that the base image (check in the container registry where the image is hosted) used is available for your target architectures and scripts used to build images or artifacts should compile and install the software corresponding to the target platform in the container.
Step 1: Set up container registry secrets
To add the repository secrets, go to the GitHub repository and navigate to Settings > Secrets > Actions > New repository secret to add repository secrets.
Note: REGISTRY_USERNAME and REGISTRY_PASSWORD secrets contain credentials for pushing container image from the GitHub Actions workflow to your container registry (quay.io).
Step 2: Configure GitHub Actions workflow
The GitHub Actions workflow uses official Docker Actions available in the GitHub Marketplace, to facilitate the process. To configure the GitHub Actions workflow, add the workflow YAML file (build_image.yaml) to the .github/workflows directory in your GitHub repository.
Note: If you are new to GitHub Actions, go through the official documentation to understand the basic workflow structure.
The following code snippet indicates steps to configure the GitHub Actions workflow:
- docker/setup-qemu-action and docker/setup-buildx-action sets up QEMU and Docker Buildx to cross compile the image for IBM Power.
- docker/login-action uses the secrets configured in Step 1: Set up container registry secrets to authenticate with the container registry.
- docker/build-push-action builds the image and provides the platforms option to specify the target architecture. It provides several options to customize the docker builds. The advantage of using docker/build-push-action over the Docker Buildx commands is that it automatically handles architecture tagging, pushing, and creating the manifest file for multi-arch images.
name: Build Multi-arch Image
on: [push, workflow_dispatch]
jobs:
build:
name: Build image
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to container registry
uses: docker/login-action@v2
with:
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_TOKEN }}
registry: quay.io
- name: Build and Push Image
uses: docker/build-push-action@v3
with:
tags: quay.io/siddhesh_ghadi/multi-arch-gha:latest
platforms: linux/amd64,linux/ppc64le
push: true
If you do not want to use docker/build-push-action, you can also build the image using the following configuration:
- name: Build & Push Image
run: |
docker buildx build --push --platform linux/amd64,linux/ppc64le --tag quay.io/siddhesh_ghadi/multi-arch-gha:latest .
Step 3: Run GitHub actions workflow
The following steps explain how to run a GitHub Actions workflow:
- The GitHub Actions workflow is configured to trigger a run on each push commit in repository.
- Click Build image to view the logs.
- Upon completion, the workflow pushes the multi-arch container image to the quay.io image registry.
Note: You can push the image to any container registry, for example, gcr.io, DockerHub, etc.
Step 4: Verify multi-arch container image
The following steps confirm that the container image that is compatible with x86 is also compatible with IBM Power.
- The following command verifies the compatibility of the container image on the x86 architecture.
root@x86:~# docker pull quay.io/siddhesh_ghadi/multi-arch-gha:latest
latest: Pulling from siddhesh_ghadi/multi-arch-gha
a96e4e55e78a: Pull complete
67d8ef478732: Pull complete
048ed16a298a: Pull complete
Digest: sha256:8a2ac0d6ec4edd07ca080bf5fcce6f526f58ba3ba4774c684ef8db988764cb93
Status: Downloaded newer image for quay.io/siddhesh_ghadi/multi-arch-gha:latest
quay.io/siddhesh_ghadi/multi-arch-gha:latest
root@x86:~# docker inspect quay.io/siddhesh_ghadi/multi-arch-gha:latest|grep Arch
"Architecture": "amd64",
root@x86:~# docker run --rm quay.io/siddhesh_ghadi/multi-arch-gha:latest
Container for x86_64 CPU architecture.
- The following command verifies that the container image is compatible on the IBM Power architecture, hence establishing multi-arch support with GitHub Actions.
[root@ppc64le ~]# docker pull quay.io/siddhesh_ghadi/multi-arch-gha:latest
latest: Pulling from siddhesh_ghadi/multi-arch-gha
77d8a8d64c34: Pull complete
c7a5b2f22050: Pull complete
5a8efc2bee46: Pull complete
Digest: sha256:8a2ac0d6ec4edd07ca080bf5fcce6f526f58ba3ba4774c684ef8db988764cb93
Status: Downloaded newer image for quay.io/siddhesh_ghadi/multi-arch-gha:latest
quay.io/siddhesh_ghadi/multi-arch-gha:latest
[root@ppc64le ~]# docker inspect quay.io/siddhesh_ghadi/multi-arch-gha:latest|grep Arch
"Architecture": "ppc64le",
[root@ppc64le ~]# docker run --rm quay.io/siddhesh_ghadi/multi-arch-gha:latest
Container for ppc64le CPU architecture.
Drawbacks of cross-builds
Although, cross-compilation is generally a useful strategy, it has the following drawbacks:
- Builds using the emulator are typically slower than building natively, mainly because the emulator needs to convert all instructions to the target CPU architecture.
- A build might sometimes fail due to incorrect emulation of some low-level code. In such cases moving to native hardware is the best option. For example, native assembly code that may depend intrinsically on page size (4k/x86-64, 64k/ppc64le).
- Cross-compiled artifacts are often poorly optimized because the emulator is executing code for one architecture on a different CPU and thus loses some of the hardware acceleration benefits.
Summary
To summarize:
- In the first part you cross-compiled and built an IBM Power image on a local x86 host using Docker Buildx (locally).
- In the second part, you performed similar steps with the help of GitHub Actions workflow establishing multi-arch support in a CI/CD environment.
- Later, you tested the container on a native machine to ensure that cross-compilation worked successfully.
- And finally, we listed a few cross-compilation drawbacks.
If you want to include testing on a native machine, you can set up a GitHub Actions self hosted runner and include testing of your container.
Give it a try and let us know how it went in the comments section below. If you need access to a Power VM, take a look at the resources listed in this blog for several options, many of them are free.
#Featured-area-1
#Featured-area-1-home