How RHEL AI on IBM Cloud Can Support Custom Model Tuning for Generative AI Transformation

By Dan Waugh posted Tue May 07, 2024 10:00 AM

Like

By: Dan Waugh &

Dinakaran Joseph

While AI has been around for decades, it’s only been a little more than a year since the ChatGPT and Gen AI buzz took over the tech world. At first, we were all amazed at the speed at which we were able to derive clear, relevant responses to a slew of questions. But over time it became clear that these responses lacked critical information needed to drive business transformation. What was missing was ‘fine tuning’ – the process of adapting pre-trained models for specific use cases or tasks.

This is why Red Hat has introduced Red Hat Enterprise Linux AI (RHEL AI): a foundation model platform that enables users to more seamlessly develop, test and deploy generative AI (GenAI) models

We believe that RHEL AI will help address several key challenges to leveraging Gen AI, and can help tackle fine-tuning for your enterprise. Over time we plan to have these images natively available on IBM Cloud as well.

RHEL AI platform for model fine-tuning and inferencing

Implementing an AI strategy requires more than simply selecting a model. Enterprises need the expertise to tune a given model for their specific use case, as well as deal with the significant costs of AI implementation.:

Procuring AI infrastructure or consuming AI services
The complex process of tuning AI models for specific business needs
Integrating AI into enterprise applications
Managing both the application and model lifecycle

Red Hat is addressing these requirements by bringing together IBM and strategic industry GPU partners to create an integrated toolset for Gen AI fine tuning. This all-in-one bootable image is called RHEL AI, and includes:

Built-in GPU/Accelerator support for NVIDIA, AMD and Intel solutions
Built-in open source version of IBM’s Granite LLM, IBM's flagship series of LLM foundation models trained on trusted enterprise data spanning internet, academic, code, legal and finance
InstructLab CLI, IBM’s novel solution to facilitate Large Language model development & tuning through collaboration among the open-source community.

Using simple YAML language, developers can use this integrated toolset to update the model with their business specific data and skills, and confidently move them to production.

The only missing piece is infrastructure: The on-demand GPU, CPU, storage, and networking to run the image for fine tuning and inferencing, plus the surrounding services to make it a complete enterprise solution. One excellent choice for that available today is IBM Cloud.

Integrating in IBM Cloud future for usage

IBM Cloud is delivering a complete set of Gen AI services with watsonx platform and AI Assistants, plus the necessary support services. As part of this integrated stack for AI workloads, IBM Cloud is positioned as the most complete, secure and compliant cloud for building Gen AI solutions and infusing Gen AI into co-located enterprise applications, enabling ecosystem partners and clients to integrate Foundation Models in their AI journey.

This same infrastructure will support an on-demand support for RHEL AI images, providing an immediate, flexible, and cost-effective set of GPU scaling options that yet you choose the best fit, from initial trail all the way a continuous delivery pipeline.

Starting with Red Hat’s announced the RHEL AI developer preview on May 7, 2024, you can use IBM cloud to run the models and train with your own skills and data with the InstructLabs CLI. Developers can build their own bootable images and bring them to IBM Cloud as custom VPCI (Virtual Private Cloud) images; over coming months IBM will also be offering standard RHEL AI images that are up to date with the community contributions.

Try it yourself today: Step by step walk through for RHEL AI on IBM Cloud VPC:

The RHEL AI developer preview is open for users to build their own images today, and IBM Cloud VPC is the perfect landing zone for complete cloud-based resources. Here’s step by step instructions on launching your own instance on IBM Cloud.

Follow the instructions in “RHEL AI Dev Preview Guide” to create the bootc containers.
The Red Hat Enterprise Linux AI is currently available as a Developer Preview that you build yourself and then boot on IBM Cloud. Red Hat’s guide for this build is posted at: https://github.com/RedHatOfficial/rhelai-dev-preview
Important: As part of the “Creating bootc containers” section of the document, before pushing the bootc container to the registry you need to issue the following additional build commands:
1. # install cloudinit by issuing the make command
  make cloud-nvidia
2. # Point to your registry:
  export REGISTRY_ORG=<your-quay.io-user-name>
3. #Create a qcow2 disk image from the bootc container:
  make disk-nvidia BOOTC_IMAGE= quay.io/rhelai-dev-preview/nvidia-bootc-cloud:latest
Upload the qcow2 image to an IBM COS bucket using the IBM Cloud CLI command:
ibmcloud cos object-put --bucket <bucket name> --key <name of the image>.qcow2 --body ./disk.qcow2
Add access to COS for the Image creation service using the IBM Cloud CLI:
ibmcloud iam authorization-policy-create is cloud-object-storage Reader --source-service-account <Account ID> --source-resource-type image
Create a VPC image from the qcow2 image using the IBM Cloud CLI:
ibmcloud is image-create <name of vpc image> --file <file in cos> --os-name red-9-amd64-byol
Deploy an IBM Cloud VPC VSI with the gx3 profile using the custom image created above, making sure to correctly configure your SSH credentials at the same time.
Continue with the “Using RHEL AI and InstructLab” instructions in the Dev Preview guide.

Where to go next…

Customizing a trusted Gen AI model for your organization is a start of a journey. Starting with the RHEL AI platform on IBM Cloud can be your fastest path to demonstrating positive results for your organization: Simplifying processes, adding clarity, subtracting roadblocks.

Let’s look at the way forward from here….

Build your Expertise on MML Fine Tuning:
Lean-in to fine tuning with IBM’s open-source InstructLabs community at https://github.com/instructlab . Here you can l expand your skills and contribute to community as well as for your business.
Scale up solutions to enterprise level with OpenShift AI
As you scale your teams will naturally outgrow single GPU deployments like RHEL AI. A great next step is to leverage the power of OpenShift on IBM Cloud to accelerate your Gen AI development with scalable clustering solutions featuring OpenShift AI. To get started with your own scaled-up version and learn more, visit https://cloud.ibm.com/docs/openshift?topic=openshift-datascience.
Move to full As-a-Service with watsonx.ai
IBM watsonx.ai provides a studio of integrated tools for working with generative AI capabilities that are powered by foundation models and for building machine learning models. These are all provided as a service and integrated with the IBM Cloud platform for complete application deployments at scale. For more information, visit https://www.ibm.com/docs/en/watsonx/saas?topic=overview-watsonx

Cloud Platform as a Service