CircleCI is the continuous integration and continuous delivery (CI/CD) platform that helps development teams to release code rapidly and automate build, test, and deploy. After repositories on GitHub or Bitbucket are authorized and added as a project on circleci.com, every code triggers CircleCI jobs.
CircleCI offers several execution environments including, Docker, Linux VM (virtual machine), MacOS, Windows, GPU, and Arm. Each job defined in your project configuration is run in a separate execution environment, either a Docker container or a virtual machine. If you need a different operating system or require a specific hardware configuration, you can host your own runners. CircleCI self-hosted runner enables you to use your own infrastructure for running jobs. This allows us to build and test on a wider variety of architectures, as well as have additional control over the environment. Currently, CircleCI supports self-hosted runners on the platforms mentioned on the CircleCI - Runner Supported Platforms page.
In this tutorial, we will see how to build multi-arch images by using self-hosted runners in CircleCI. Since ppc64le self-hosted runners are not supported, we will use a workaround to SSH into the Power machine from the self-hosted x86 runner.
Pre-requisites:
- A user account on GitHub
- An x86 VM for adding a self-hosted runner to the CircleCI
- An ppc64le VM for executing the CircleCI job
If you don't have access to your own ppc64le VM, you can use the PowerVS service at IBM Cloud get one. This example uses both CentOS 8 VMs.
- Install Podman and Buildah on the x86 and ppc64le VMs:
sudo yum -y install podman buildah
Step 0: Create accounts on the basic services required
We are using GitHub to host the source code, CircleCI as our CI tool, and Quay.io to host our multi-arch image, so if you want to follow the steps described here, you must have an account on each service.
Step 1: Setup the Quay repository with a new Robot account
Create a repository on Quay.io and add a Robot account to it. A robot account will be useful to pull/push your multi-arch images from/to the container registry (ensure you set the write permission to your robot, like the picture below).
Step 2: Generate SSH key
Create SSH key pair if it doesn’t exist by using the following steps on the x86 VM.
cd ~/.ssh
ssh-keygen -o -t rsa
Press Enter to complete with the default configurations, we will leave the passphrase empty for now but, you can use it to add extra security to your key.
When you type ls you should find two files: id_rsa and id_rsa.pub.
id_rsa has the private key while id_rsa.pub has the public key.
Step 3: Add public ssh key to the authorized_keys file of a ppc64le VM
First view/copy the contents of your recently generated public key id_rsa.pub on the x86 machine. The public ssh key begins with "ssh-rsa" and ends with your email address:
cat ~/.ssh/id_rsa.pub
Login to your ppc64le VM and editauthorized_keys file by putting the contents of your public key below any other keys in that file:
vi ~/.ssh/authorized_keys
Step 4: Test SSH connection using the private key
Just to make sure that the public ssh key is added correctly to the authorized_keys file of a ppc64le VM, try SSHing into the ppc64le machine from the x86 machine with the corresponding private key.
ssh -i /root/.ssh/id_rsa username@hostname
Step 5: Setup the Github repository
Create a new repository or simply fork this repository into your GitHub account. Make sure that you are following the hosted-multi-arch branch of my GitHub repository.
Step 6: Set up your Build on CircleCI
- For this step, you will need a CircleCI account. Visit the CircleCI signup page and click “Sign Up with GitHub”. You will need to give CircleCI access to your GitHub account to run your builds. If you already have a CircleCI account then you can navigate to your dashboard.
- Next, you need to add your repo as a new project on CircleCI. To add your new repo, ensure that your GitHub account is selected in the dropdown in the upper-left, find the repository you just created below, and click the Setup project button next to it.
- On the redirected page, you will notice three options (Fastest, Faster and Fast), with Fastest selected as the default. We’ll use this default option.
- Enter hosted-multi-arch in the input field for the GitHub branch (notice the text underneath the field confirming the presence of the
.circleci/config.yml file) and click Set Up Project.
Within no time, the build will get failed, as we are yet to add the self-hosted runner and the credentials associated with the Quay repository as an environment variable in our project.
Step 7: Add a self-hosted runner to CircleCI
Follow these instructions for adding a self-hosted runner to CircleCI on an x86 machine. Once your runner is successfully configured and listening for jobs, it will display as idle which means you can execute your workflow.
You will need to update your custom resource class name of Self-Hosted Runner in .circleci/config.yml present on the GitHub repository.
Step 8: Set an environment variable in a project
To customize our builds and avoid exposing secrets we will create environment variables on a CircleCI Project.
- On the CircleCI web app, click the Project Settings button on the project’s individual Pipelines page.
- Click on Environment Variables in the side navigation.
- Click the Add Variable button to enter the name and value of the new environment variable.
GH_REPO : Name of your GitHub Repo
PPC64LE_MACHINE_IP : IP of ppc64le
VMQUAY_REPO : Quay repository url for publishing images eg. quay.io/<user>/<repository-name>
QUAY_USER : Quay Robot user
QUAY_PASS : Quay Robot token
Step 9: Re-trigger the CircleCI build
To use the newly created Environment Variables, re-trigger the CircleCI build by committing the .circleci/config.yml file or by re-running the failed workflow from the start.
Once the workflow is executed successfully, verify that the multi-arch image is pushed to your quay repository.
That’s all folks! Thanks for reading. Hope this tutorial was helpful.
Originally published on Medium