This blog details the steps required to run inferencing with ONNX Runtime on IBM Power10 systems using a resnet model. A resnet model is a deep neural network architecture designed to combat the vanishing gradient problem, allowing for the effective training of very deep networks. Resnet models are used for various computer vision tasks and this blog will demonstrate its image detection capabilities.

Prerequisites

This blog assumes the user already has conda installed. Utilize the following blog post by Sebastian Lehrig to get conda setup on power if needed.

Environment Setup

Create a new conda environment.

conda create --name your-env-name-here python=3.11

This will create a new environment and install python version 3.11 and its required dependencies.

Activate the newly created environment.

conda activate your-env-name-here

Once the environment is active, install openblas, onnxruntime, and their dependencies.

conda install libopenblas -c rocketce

conda install onnxruntime -c rocketce

conda install pillow -c rocketce

When using the conda install command with the -c argument, packages will attempt be installed from a specified channel. Packages installed via the rocketce channel will have MMA optimizations.

Project Setup

Navigate to a desired project directory and download the model from the ONNX Model Zoo.

wget https://github.com/onnx/models/raw/main/validated/vision/classification/resnet/model/resnet50-v1-12.onnx

Download the ImageNet Labels.

wget https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt

Optionally, download a stock dog image to use for classification.

wget https://raw.githubusercontent.com/pytorch/hub/master/images/dog.jpg

Create a new python script inside the project directory.

touch resnet.py

Open the python script with any text editor or IDE (vi, vim, nano, vscode, etc…) and paste the following code.

import numpy as np
import onnxruntime
import urllib
import urllib.request
import os
import argparse
from PIL import Image

# Classify an image using a pretrained resnet model
def classifyImage(image, threads, debug):
    # Check input parameters
    if image is not None:
        # Ensure that the provided image exists
        if checkImagePath(image):
            input_image = Image.open(image)
            if debug: print(f'Got custom image at {image}')
        else:
            print('Image not found. Check the provided path.')
            exit()
    else:
        # Use stock dog image if custom image was not given
        url, filename = ("<https://github.com/pytorch/hub/raw/master/images/dog.jpg>", "dog.jpg")
        try: urllib.URLopener().retrieve(url, filename)
        except: urllib.request.urlretrieve(url, filename)
        input_image = Image.open(filename)
        if debug: print(f'No custom image provided. Using stock image.')

    # Create session options
    opts = onnxruntime.SessionOptions()
    if threads is not None:
        if debug: print(f'Setting intra_op_num_threads and inter_op_num_threads to {threads}')
        opts.intra_op_num_threads = int(threads)
        opts.inter_op_num_threads = int(threads)

    # Create inference session
    if debug: print(f'Creating inferencing session.')
    sess = onnxruntime.InferenceSession('resnet50-v1-12.onnx', sess_options=opts)
    input_name=sess.get_inputs()[0].name
    input_tensor = preprocess(input_image)

    # Run inferencing
    if debug: print(f'Running inferencing session.')
    pred_onnx=sess.run([], {input_name: input_tensor})

    # Print results
    with open('imagenet_classes.txt') as f:
        categories = [s.strip() for s in f.readlines()]
    preds = np.squeeze(pred_onnx[0])
    preds = np.exp(preds) / np.sum(np.exp(preds)) # softmax
    a = np.argsort(preds)[::-1]
    print(categories[a[0]], preds[a[0]])

# Ensure that the user provided image path exists
def checkImagePath(imagePath):
    return os.path.exists(imagePath)

# Preprocess image according to resnet guidelines
def preprocess(img):
    img = img.resize((256, 256), Image.BILINEAR)
    img = np.array(img)
    img = img / 255.
    h, w = img.shape[0], img.shape[1]
    y0 = (h - 224) // 2
    x0 = (w - 224) // 2
    img = img[y0 : y0+224, x0 : x0+224, :]
    img = (img - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]
    img = np.transpose(img, axes=[2, 0, 1])
    img = img.astype(np.float32)
    img = np.expand_dims(img, axis=0)
    return img

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('-i', '--image', help='Absolute path to image', required=False)
    parser.add_argument('-t', '--threads', help='Thread count for execution. Sets intra_op_num_threads and inter_op_num_threads args of session_options.', required=False)
    parser.add_argument('-d', '--debug', help='Enable debug mode', required=False, action='store_true')
    args = parser.parse_args()

    classifyImage(args.image, args.threads, args.debug)

This script has the option to pass in script parameters to specify an image to classify, the number of threads to use in the session, and debug mode. These parameters are optional and work as follows.

• use -i/--image followed by an absolute path to an image to attempt to classify that image instead of the stock dog image
• use -t/--threads to specify the number of threads used for execution
• use -d/--debug to enable debug mode, this will print out intermediate results

Note: Each of these parameters are optional, the script will use a stock image and default thread settings if no parameters are provided. An example invocation can be seen as follows.

Execution

Once the script is complete, run the model and view the results.

python3 onnx_resnet.py -i /abs/path/to/image -t 1 -d

The expected output is of the form “Class Label” “Confidence/Probability”

Sample output from resnet.py:

Samoyed 0.8846219182014465