For quite some time now, the Docker community has been grappling with the complexity of supporting multiple operating systems and architectures. The following are two key issues that have arisen as a result of this diversity:
- New Docker users tend to assume that if they do a Docker run of an image, that it will work on their platform, and have no idea how to interpret the failure. Sometimes, the error message is exec format error, which is not intuitive. However, what a user may see when using the wrong type of image also depends on the image that is being run.
- Projects that do support multiple platforms have resorted to using image names to indicate which platform an image was built for. For example, my-cool-app-ppc64le:latest; or creating a Docker hub namespace for each architecture, such as ppc64le/my-cool-app.
Image Index, aka Manifest List
To alleviate these issues, the Docker community has come up with what is currently called a manifest list, also nicknamed a multi-arch image, or fat manifest. An individual manifest describes the contents of one image, and a manifest list enables you to group multiple images together. After the creation of a manifest list, anyone (from the project owners to their user-base) can use the manifest list name where they once used an individual image name. When a user runs a docker pull or a docker run command, the Docker engine does the work of selecting which image to pull based on the operating system and architecture on which it is running. As a result, a project's users no longer have to worry about finding the name of the image that will work for their platform. The project can have a single name, my-cool-app:latest , to put in blog posts, documentation, and so on. This makes the user experience much more pleasant and intuitive.
The community was quick to create tooling around the manifest list before Docker decided on how to best organize the UI. Because of all the work that was done by members of the community, the usage for such a tool began to become clear. There is currently a stand-alone tool maintained by Phil Estes from IBM which most people have been using to create multi-arch images — but the community also began to ask more and more for the ability to make multi-arch images using Docker's tooling. As a result, the manifest subcommand was born. The PR has been merged, and Docker manifest is now included in the CLI as experimental. It is also included in Docker for Mac Edge.
Here is how to use the new Docker CLI subcommand to create manifest lists.
Terminology:
- Manifest – Describes a single image.
- Image Index, or Manifest list – Groups multiple images; can be thought of as a list of pointers; must exist on a Docker registry.
- Reference – A name for an image or manifest list (for example, docker.io/library/my-cool-app:latest).
- Tag – The portion of the reference after the colon. This is not required. However, you can use it to version a reference. For example, app:v1. latest is the default tag.
- Digest – the hash (currently sha256) of the image contents of a docker image or manifest list's individual digests.
When working with multi-arch images , you can use an untagged, tagged, or a digested reference (for example, my-cool-app@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2) for image names.
Creating a manifest list
There are two ways to create a manifest list. You can interactively create, annotate (optional), and push a manifest list from your local host by using the following sequence of commands:
docker manifest create new-list-ref-name image-ref [image-ref...]
docker manifest annotate new-list-ref-name image-ref --os linux --arch arm
docker manifest push new-list-ref-name
This will store some metadata locally until you perform the final push step to the registry, at which point the manifest list will be usable. The first parameter when working with multi-arch images is always the manifest list name that you are going to give to users. The following image-ref parameters are the pre-built, platform-specific images you have previously pushed to a registry.
Note: This is an important detail to understand. Pushing a multi-arch image to a registry does not push the image layers. It only pushes a list of pointers to accessible images. This is why it is better to think of a multi-arch image as what it really is: a manifest list. Project maintainers might be the only ones who need to understand this distinction. End users of the images do not need to be concerned with the difference because the goal of a manifest list is to be transparent to the user.
Note: For insecure registries (such as a local one used for testing), if you are building and using the CI PR, ensure that you add the registry to your local Docker configuration file.
> cat ~/.docker/config.json
{
"insecure-registries" : ["127.0.0.1:5000"]
}
Here is an example of using the interactive approach to create a multi-arch image:
$ docker manifest create awesome_company/mycool-app:v1 awesome_company/mycool-app-x86:v1 \
awesome_company/mycool-app-s390x:v1 \
awesome_company/mycool-app-ppc64le:v1 \
awesome_company/mycool-app-armhf:v1
$ docker manifest annotate awesome_company/mycool-app:v1 awesome_company/mycool-app-armhf:v1 --os linux --arch arm
$ docker manifest push awesome_company/mycool-app:v1
After you have pushed your manifest list to a registry, you use it just as you would have previously used an image name.
For example, a user on an x86 system:
$ docker run awesome_company/mycool-app:v1 echo hi
hi
This command pulls and runs only the x86 image layers that your manifest list points to, and the user will have no awareness that ‘awesome_company/mycool-app:v1’ was a manifest list instead of an image name.
If you are using IBM® Power Systems™, or any other platform that you included in your manifest list, the command is exactly the same:
$ docker run awesome_company/mycool-app:v1 echo hi
hi
As you can see, multi-arch images (also known as manifest lists) greatly improve the end-user experience, with just a little extra work from the project maintainers.
Resources
#OpenSourceDevelopment