Data and AI on Power

 View Only

How to Run a GPT Model on IBM Power10 Using PyTorch

By Daniel Schenker posted Fri February 16, 2024 04:02 PM

  

This blog details the steps required to run inferencing with PyTorch on IBM Power10 systems using a gpt model. GPT stands for Generative Pre-trained Transformer and is an advanced language model trained on vast amounts of text data using unsupervised learning. This enables the model to generate coherent and contextually relevant text given a prompt.

Prerequisites

This blog assumes the user already has conda installed. Utilize the following blog post by Sebastian Lehrig to get conda setup on power if needed.

Environment Setup

Create a new conda environment.

conda create --name your-env-name-here python=3.11

This will create a new environment and install python version 3.11 and its required dependencies.

Activate the newly created environment.

conda activate your-env-name-here

Once the environment is active, install openblas, pytorch, transformers, and accelerate.

conda install libopenblas -c rocketce

conda install pytorch-cpu -c rocketce

conda install transformers -c rocketce

pip install accelerate

When using the conda install command with the -c argument, packages will attempt be installed from a specified channel. Packages installed via the rocketce channel will have MMA optimizations.

Project Setup

Navigate to a desired project directory and create a new python script.

touch gpt.py

Open the python script with any text editor or IDE (vi, vim, nano, vscode, etc…) and paste the following code.

import torch
from transformers import GPTJForCausalLM, AutoTokenizer

# Set the device based on the system being used
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Download the model and set the device
model = GPTJForCausalLM.from_pretrained("EleutherAI/gpt-j-6B", low_cpu_mem_usage=True)
model.to(device)

# Download the tokenizer and apply settings
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B")
tokenizer.padding_side = "left"
tokenizer.pad_token = tokenizer.eos_token

# Sample input text prompts
texts = ["Once there was a man ", "The weather today will be ", "The best football player in the world is ", "My next vacation is", "The Belgian national football team ", "Once there was a man ", "The weather today will be ", "The best football player in the world is ", "My next vacation is", "The Belgian national football team ", "When I was in  school", " My first day of office", " I am scared of", "Life is a dream", "What is credit by exam?", "weather prediction for December","Once there was a man ", "The weather today will be ", "The best football player in the world is ", "My next vacation is", "The Belgian national football team ", "Once there was a man ", "The weather today will be ", "The best football player in the world is ", "My next vacation is", "The Belgian national football team ", "When I was in  school", " My first day of office", " I am scared of", "Life is a dream", "What is credit by exam?", "weather prediction for December","Once there was a man ", "The weather today will be ", "The best football player in the world is ", "My next vacation is", "The Belgian national football team ", "Once there was a man ", "The weather today will be ", "The best football player in the world is ", "My next vacation is", "The Belgian national football team ", "When I was in  school", " My first day of office", " I am scared of", "Life is a dream", "What is credit by exam?", "weather prediction for December","The Belgian national football team ", "Time just flies"]

# Encode input
encoding = tokenizer(texts, padding=True, return_tensors='pt').to(device)

# Run the model
with torch.no_grad():
    generated_ids = model.generate(**encoding, do_sample=True, temperature=0.9, max_length=50)

# Decode model output
generated_texts = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)

# Print results
for text in generated_texts:
    print("---------")
    print(text)

Execution

Once the script is complete, run the model and view the results.

python3 gpt.py

Conclusion

This blog detailed the steps required to run inferencing with PyTorch on IBM Power10 systems using a gpt model. Batch sizes can be changed by modifying “texts” list in the above python script.

Permalink