The IBM Linux on Power team is thrilled to announce the availability of optimized wheels for Python! While Python is theoretically an architecture-independent language, most know that the distribution packages for wheels often include optimized components built with other languages. Starting now, IBM is building a subset of these open source Python packages natively on Power for you and making them available via a DevPi server. These wheels have been selected based on various AI projects in progress around the Power brand and should represent a healthy base set of packages for anyone looking to do Python based development natively on IBM Power9, IBM Power10, and IBM Power11 servers.
There are a couple of other blogs with details on how to get access, so here we'll just include the basics for now to get you going. The very first is that we'll strongly recommend using Python 3.10, 3.11 or 3.12, and within those, you'll typically see Python 3.11 and 3.12 currently have the greatest number of packages and versions available. We also highly recommend using $ python3.12 -m venv venv to create and use a local virtual environment when working with pip. Pip's dependency resolver attempts to work major magic, and it is simplest to contain it to a virtual environment to minimize the number of dependencies it might find and constraints it tries to work through. Once you have a virtual environment, we strongly recommend the use of the pip flag --prefer-binary which tells the dependency resolver to prefer applications which have been compiled to work on this platform, which ensures that it will work to pull from our wheel repository.
The wheel repository is located at: https://wheels.developerfirst.ibm.com, and a pip install line might be:
$ pip install –prefer-binary vllm –extra-index-url=https://wheels.developerfirst.ibm.com/ppc64le/linux
Note that –extra-index-url specifies where to look in addition to pypi.org. Any noarch dependencies are fetched from pypi.org in this case.
The server is a DevPi server. In addition to accessing the wheels as you normally would with pip, you can also install devpi-client locally (it is pip/pipx installable) and then run:
$ devpi use https://wheels.developerfirst.ibm.com/ppc64le/linux
$ devpi list
And devpi will return a list of all packages in the repository. This set of packages will typically be able to resolve against most major packages either with Power specific package in this repo or from the noarch packages from pypi.org. However, if you have issues where something does not resolve correctly, drop us a comment on the TechXchange community and we'll investigate it. We can't solve issues that aren't solved on PyPI, and sometimes the constraints will get complex, but we do have some practice and may be able to help untangle or provide a fix.
You might wonder why we created a whole new repository rather than updating the packages on pypi.org to directly support Power. That's an excellent question, and our gold standard is to be fully integrated upstream. However, that level of integration has a number of challenges which we are working through as some of you have already seen. For instance, we clearly need a well-integrated, upstream CI/CD, and for that we recently added GitHub Actions for Power (and IBM Z and LinuxOne!) to our options, it will take a while to get upstream open source migrated. Also, managing versions, optimizations, testing, dependencies, etc. are all activities that take either our team or upstream/our team bandwidth, and that is always done with the community in collaboration, which can take time. In short, to provide a broader set of access quickly, we felt it easiest to set up a repository for wheels.
We expect this list to evolve over time, generally based on active AI projects. While we aren't formally set up for taking requests, you are welcome to put suggestions in the comments here where we will occasionally review and determine what additional packages or versions we may add. We are hoping to add support for Python 3.13 in due time, depending on when Python 3.13 is released with RHEL. Also, additional versions may be added based on our needs.
Our goal is to enable Power customers to use Python and evaluate how they can get the most value from their systems. The blending of the large data that Power servers typically have access to, its ability to process massive amounts of data, and Python and optimized libraries should provide some steppingstones for data analysis, deep learning, machine learning, generative AI and agentic AI.
This is a new, first of a kind activity for us and we are really interested in your feedback. Drop us a line about what you will do with Python on IBM Power.
Check out these blogs to know what our incredible development team has been doing:
Many thanks to our development team that built the infrastructure and completed the ports of these packages to Power.
------------------------------
Gerrit
------------------------------