Cloud Platform as a Service

Cloud Platform as a Service

Join us to learn more from a community of collaborative experts and IBM Cloud product users to share advice and best practices with peers and stay up to date regarding product enhancements, regional user group meetings, webinars, how-to blogs, and other helpful materials.

 View Only

Use the Red Hat AI Inference on IBM Cloud service to back Red Hat OpenShift Lightspeed in a Red Hat OpenShift on IBM Cloud cluster

By Darrell Schrag posted 06/09/26 03:23 PM

  

Red Hat OpenShift Lightspeed is a generative AI-based virtual assistant for Red Hat OpenShift users of all skill levels. Using natural language, users can ask OpenShift Lightspeed questions about Red Hat OpenShift and get step-by-step guidance from it directly within the Red Hat OpenShift web console.

OpenShift Lightspeed uses the official Red Hat OpenShift documentation to generate answers and is powered by large language models (LLMs) served by your organization’s choice of provider. OpenShift Lightspeed can help improve productivity for experienced users and empower new users by helping them learn Red Hat OpenShift faster.

In this post, we will install OpenShift Lightspeed in an OpenShift cluster and back it with the Red Hat AI Inference on IBM Cloud service.

Setup a Red Hat AI Inference Service Project

Our first step is to setup a Red Hat AI Inference service project. From the project, you are going to want to save away the base URL for use later in the Lightspeed setup.

Inferencing Project

Get an API Key

Follow the instructions to create a Service ID and API Key for your Red Hat AI Inference service. Save that API key for use later.
Setup an API Key

Explore the Model Catalog

Take a look at the available models in the Red Hat AI Inference service model catalog. Determine what model you want to use as your LLM for OpenShift Lightspeed. Save that model name for later.
Model Catalog

Install OpenShift Lightspeed

Over to your Red Hat OpenShift on IBM Cloud cluster. Open the OpenShift web console and go to the Operator Hub. Find the OpenShift Lightspeed operator and install it.
Lightspeed Operator
Once the operator has installed, you will have a new OpenShift project, openshift-lightspeed, and the operator will show a successful status.
Installed Operators

Add the Red Hat AI Inference Service API Key as a Secret

We need to create a Kubernetes Secret to store the Red Hat AI Inference Service API Key that we created above. The name of the secret here is credentials, the secret is stored in the openshift-lightspeed namespace, and we provide our API key as the value for the apitoken key. 
apiVersion: v1
kind: Secret
metadata:
  name: credentials
  namespace: openshift-lightspeed
type: Opaque
stringData:
  apitoken: <api_key>

Create the OLSConfig

The OLSConfig is where we wire up the Red Hat AI Inference service to OpenShift Lightspeed. Below are a few chunks of the YAML to see how to configure the OpenShift Lightspeed configuration.
The spec part of the yaml contains the following pertinent information:
  • the credentialsSecretRef points to the secret we created above, named credentials
  • the model name is the model we chose from the Red Hat AI Inference service model catalog, granite-4-0-h-small
  • the name we gave this llm provider is 'Red Hat AI Inference'
  • the provider type is rhoai_vllm since the Red Hat AI Inference service is backed by vLLM

spec:
  llm:
    providers:
      - credentialsSecretRef:
          name: credentials
        models:
          - name: granite-4-0-h-small
            parameters:
              toolBudgetRatio: 0.5
            url: 'https://us-east.rhai.ibm.com/v1/projects/<unique ID>/inference'
        name: Red Hat AI Inference
        type: rhoai_vllm
        url: 'https://us-east.rhai.ibm.com/v1/projects/<unique ID>/inference'

The ols portion of the yaml contains two important values:

  • the defaultModel references our chosen model from the model catalog
  • the defaultProvider references the provider we specified in the spec section

defaultModel: granite-4-0-h-small
defaultProvider: Red Hat AI Inference

It takes a few minutes for the OLSConfig to finish and show a successful status. 

Now that we have everything setup, we can try out OpenShift Lightspeed. Open Lightspeed from the button in the lower right hand corner of the OpenShift console.

Lightspeed button
The Lightspeed window is now ready for us to ask an OpenShift question. 
Give it a try.
Lightspeed
For more information about the Red Hat AI Inference service, consult this blog. Also check out the Red Hat documentation.
Darrell Schrag - product manager Red Hat OpenShift on IBM Cloud
0 comments
12 views

Permalink

https://community.ibm.com/community/user/blogs/darrell-schrag/2026/06/09/use-the-ibm-cloud-inferencing-service-to-back