This blog details the steps required to run inferencing with ONNX Runtime on IBM Power10 systems using a resnet model. A resnet model is a deep neural network architecture designed to combat the vanishing gradient problem, allowing for the effective training of very deep networks. Resnet models are used for various computer vision tasks and this blog will demonstrate its image detection capabilities.
Prerequisites
This blog assumes the user already has conda installed. Utilize the following blog post by Sebastian Lehrig to get conda setup on power if needed.
Environment Setup
Create a new conda environment.
conda create --name your-env-name-here python=3.11
This will create a new environment and install python version 3.11 and its required dependencies.
Activate the newly created environment.
conda activate your-env-name-here
Once the environment is active, install openblas, onnxruntime, and their dependencies.
conda install libopenblas -c rocketce
conda install onnxruntime -c rocketce
conda install pillow -c rocketce
When using the conda install command with the -c argument, packages will attempt be installed from a specified channel. Packages installed via the rocketce channel will have MMA optimizations.
Project Setup
Navigate to a desired project directory and download the model from the ONNX Model Zoo.
wget https://github.com/onnx/models/raw/main/validated/vision/classification/resnet/model/resnet50-v1-12.onnx
Download the ImageNet Labels.
wget https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt
Optionally, download a stock dog image to use for classification.
wget https://raw.githubusercontent.com/pytorch/hub/master/images/dog.jpg
Create a new python script inside the project directory.
touch resnet.py
Open the python script with any text editor or IDE (vi, vim, nano, vscode, etc…) and paste the following code.
import numpy as np
import onnxruntime
import urllib
import urllib.request
import os
import argparse
from PIL import Image
# Classify an image using a pretrained resnet model
def classifyImage(image, threads, debug):
# Check input parameters
if image is not None:
# Ensure that the provided image exists
if checkImagePath(image):
input_image = Image.open(image)
if debug: print(f'Got custom image at {image}')
else:
print('Image not found. Check the provided path.')
exit()
else:
# Use stock dog image if custom image was not given
url, filename = ("<https://github.com/pytorch/hub/raw/master/images/dog.jpg>", "dog.jpg")
try: urllib.URLopener().retrieve(url, filename)
except: urllib.request.urlretrieve(url, filename)
input_image = Image.open(filename)
if debug: print(f'No custom image provided. Using stock image.')
# Create session options
opts = onnxruntime.SessionOptions()
if threads is not None:
if debug: print(f'Setting intra_op_num_threads and inter_op_num_threads to {threads}')
opts.intra_op_num_threads = int(threads)
opts.inter_op_num_threads = int(threads)
# Create inference session
if debug: print(f'Creating inferencing session.')
sess = onnxruntime.InferenceSession('resnet50-v1-12.onnx', sess_options=opts)
input_name=sess.get_inputs()[0].name
input_tensor = preprocess(input_image)
# Run inferencing
if debug: print(f'Running inferencing session.')
pred_onnx=sess.run([], {input_name: input_tensor})
# Print results
with open('imagenet_classes.txt') as f:
categories = [s.strip() for s in f.readlines()]
preds = np.squeeze(pred_onnx[0])
preds = np.exp(preds) / np.sum(np.exp(preds)) # softmax
a = np.argsort(preds)[::-1]
print(categories[a[0]], preds[a[0]])
# Ensure that the user provided image path exists
def checkImagePath(imagePath):
return os.path.exists(imagePath)
# Preprocess image according to resnet guidelines
def preprocess(img):
img = img.resize((256, 256), Image.BILINEAR)
img = np.array(img)
img = img / 255.
h, w = img.shape[0], img.shape[1]
y0 = (h - 224) // 2
x0 = (w - 224) // 2
img = img[y0 : y0+224, x0 : x0+224, :]
img = (img - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]
img = np.transpose(img, axes=[2, 0, 1])
img = img.astype(np.float32)
img = np.expand_dims(img, axis=0)
return img
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('-i', '--image', help='Absolute path to image', required=False)
parser.add_argument('-t', '--threads', help='Thread count for execution. Sets intra_op_num_threads and inter_op_num_threads args of session_options.', required=False)
parser.add_argument('-d', '--debug', help='Enable debug mode', required=False, action='store_true')
args = parser.parse_args()
classifyImage(args.image, args.threads, args.debug)
This script has the option to pass in script parameters to specify an image to classify, the number of threads to use in the session, and debug mode. These parameters are optional and work as follows.
- use -i/--image followed by an absolute path to an image to attempt to classify that image instead of the stock dog image
- use -t/--threads to specify the number of threads used for execution
- use -d/--debug to enable debug mode, this will print out intermediate results
Note: Each of these parameters are optional, the script will use a stock image and default thread settings if no parameters are provided. An example invocation can be seen as follows.
Execution
Once the script is complete, run the model and view the results.
python3 onnx_resnet.py -i /abs/path/to/image -t 1 -d
The expected output is of the form “Class Label” “Confidence/Probability”
Sample output from resnet.py
:
Samoyed 0.8846219182014465