For quite some time now, the Docker community has been grappling with the complexity of supporting multiple operating systems and architectures. The following are two key issues that have arisen as a result of this diversity:
- New Docker users tend to assume that if they do a Docker run of an image, that it will work on their platform, and have no idea how to interpret the failure. Sometimes, the error message is exec format error, which is not intuitive. However, what a user may see when using the wrong type of image also depends on the image that is being run.
- Projects that do support multiple platforms have resorted to using image names to indicate which platform an image was built for. For example, my-cool-app-ppc64le:latest; or creating a Docker hub namespace for each architecture, such as ppc64le/my-cool-app.
Manifest list
To alleviate these issues, the Docker community has come up with what is currently called a manifest list, also nicknamed a multi-arch image, or fat manifest. An individual manifest describes the contents of one image, and a manifest list enables you to group multiple images together. After the creation of a manifest list, anyone (from the project owners to their user-base) can use the manifest list name where they once used an individual image name. When a user runs a docker pull or a docker run command, the Docker engine does the work of selecting which image to pull based on the operating system and architecture on which it is running. As a result, a project's users no longer have to worry about finding the name of the image that will work for their platform. The project can have a single name, my-cool-app:latest , to put in blog posts, documentation, and so on. This makes the user experience much more pleasant and intuitive.
The community was quick to create tooling around the manifest list before Docker decided on how to best organize the UI. Because of all the work that was done by members of the community, the usage for such a tool began to become clear. There is currently a stand-alone tool maintained by Phil Estes from IBM which most people have been using to create multi-arch images — but the community also began to ask more and more for the ability to make multi-arch images using Docker's tooling. As a result, the manifest subcommand was born. The PR has been merged, and Docker manifest is now included in the CLI as experimental. It is also included in Docker for Mac Edge.
Here is how to use the new Docker CLI subcommand to create manifest lists.
Terminology:
- Manifest – Describes a single image.
- Manifest list – Groups multiple images; can be thought of as a list of pointers; must exist on a Docker registry.
- Reference – A name for an image or manifest list (for example, docker.io/library/my-cool-app:latest).
- Tag – The portion of the reference after the colon. This is not required. However, you can use it to version a reference. For example, app:v1. latest is the default tag.
- Digest – the hash (currently sha256) of the image contents of a docker image or manifest list's individual digests.
When working with multi-arch images , you can use an untagged, tagged, or a digested reference (for example, my-cool-app@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2) for image names.
Creating a manifest list
There are two ways to create a manifest list. You can interactively create, annotate (optional), and push a manifest list from your local host by using the following sequence of commands:
docker manifest create new-list-ref-name image-ref [image-ref...]
docker manifest annotate new-list-ref-name image-ref --os linux --arch arm
docker manifest push new-list-ref-name
This will store some metadata locally until you perform the final push step to the registry, at which point the manifest list will be usable. The first parameter when working with multi-arch images is always the manifest list name that you are going to give to users. The following image-ref parameters are the pre-built, platform-specific images you have previously pushed to a registry.
Note: This is an important detail to understand. Pushing a multi-arch image to a registry does not push the image layers. It only pushes a list of pointers to accessible images. This is why it is better to think of a multi-arch image as what it really is: a manifest list. Project maintainers might be the only ones who need to understand this distinction. End users of the images do not need to be concerned with the difference because the goal of a manifest list is to be transparent to the user.
If you would like to create a multi-arch image for your app or project, you can recompile the Docker CLI and replace your current CLI binary (typically /usr/bin/docker — NOT /usr/bin/dockerd), or use the new CLI straight from the build dir of the Docker project.
See the Docker CLI docs for usage (and don't use the older version from the clnperez repo) and for how to enable an experimental CLI feature.
There is a release created that does not have the manifest subcommand marked as experimental. You can download the binary for your architecture from: https://github.com/clnperez/cli/releases/tag/1.0
For example:
sudo cp /usr/bin/docker /usr/bin/docker-cli.bak
sudo curl -fsSL https://github.com/clnperez/cli/releases/download/1.0/docker-linux-amd64 -o /usr/bin/docker
To compile in the experimental requirement:
git clone --no-checkout https://github.com/docker/cli.git && git checkout da86425e9f02a99659aeb8ecb8a93d2ee8b31e21
To build using the project’s Dockerfile if you are on an x86 system:
make -f docker.Makefile binary
To build the binary for all supported platforms (still on x86):
make -f docker.Makefile cross
The binaries (if you ran make cross there will be more than one) will be found in the build/ directory:
$ ls build/
docker docker-darwin-amd64 docker-linux-amd64 docker-linux-arm docker-linux-ppc64le docker-windows-amd64
You can use these binaries simply as:
./build/docker manifest --help
Now, you can follow the steps above (either the interactive or YAML approach) to push your multi-arch images to the registry of your choice.
Note: For insecure registries (such as a local one used for testing), if you are building and using the CI PR, ensure that you add the registry to your local Docker configuration file.
> cat ~/.docker/config.json
{
"insecure-registries" : ["127.0.0.1:5000"]
}
Here is an example of using the interactive approach to create a multi-arch image:
$ docker manifest create awesome_company/mycool-app:v1 awesome_company/mycool-app-x86:v1 \
awesome_company/mycool-app-s390x:v1 \
awesome_company/mycool-app-ppc64le:v1 \
awesome_company/mycool-app-armhf:v1
$ docker manifest annotate awesome_company/mycool-app:v1 awesome_company/mycool-app-armhf:v1 --os linux --arch arm
$ docker manifest push awesome_company/mycool-app:v1
After you have pushed your manifest list to a registry, you use it just as you would have previously used an image name.
For example, a user on an x86 system:
$ docker run awesome_company/mycool-app:v1 echo hi
hi
This command pulls and runs only the x86 image layers that your manifest list points to, and the user will have no awareness that ‘awesome_company/mycool-app:v1’ was a manifest list instead of an image name.
If you are using IBM® Power Systems™, or any other platform that you included in your manifest list, the command is exactly the same:
$ docker run awesome_company/mycool-app:v1 echo hi
hi
As you can see, multi-arch images (also known as manifest lists) greatly improve the end-user experience, with just a little extra work from the project maintainers.
Resources
