This blog outlines the process for building and running InstructLab on IBM Power10 systems. Note that this tutorial focuses on building ilab on bare metal. See Marvin Gießing's blog for instructions on running InstructLab via a prebuilt podman container.

Prerequisites

Users must have already installed a distribution of conda on a machine running Linux on Power10. See Sebastian Lehrig's guide.

Note that InstructLab depends on PyTorch, which at time of publishing does not fully support Python 3.12, so users whose conda environments are built on 3.12 may want to consider downgrading or switching environments. Python 3.11 is recommended.

1. Setup

Create a directory for InstructLab and step into it. Then download two GitHub directories (OpenBLAS and PyTorch) which we will need later.

$ mkdir instructlab
$ cd instructlab
$ git clone https://github.com/OpenMathLib/OpenBLAS.git
$ git clone --recursive https://github.com/pytorch/pytorch.git

Finally, in your chosen conda environment, install the most updated version of the necessary compilers.

$ conda install -c conda-forge gcc gxx gfortran

2. Installing PyTorch

InstructLab requires PyTorch version 2.2.1 or higher. Since no compatible binaries are available that satisfy that requirement, we will first need to need to compile PyTorch from source. To do so, we will also need to compile one of PyTorch's dependencies from source.

2a. Installing OpenBLAS

Available binaries for OpenBLAS, one of PyTorch's dependencies, were compiled using an older version of gcc, causing incompatibility. Therefore, to compile PyTorch, we must first build OpenBLAS from source. Step into your clone of the OpenBLAS repo and compile.

$ cd OpenBLAS
$ USE_OPENMP=1 make
$ cd ../pytorch

After verifying that compilation has succeeded, move to the cloned PyTorch repo.

With OpenBLAS built, we can install the rest of PyTorch's dependencies. We also point the compiler to our version of OpenBLAS by setting the OpenBLAS_HOME env variable.

$ conda install -c conda-forge astunparse numpy ninja pyyaml setuptools cmake cffi typing_extensions future six requests dataclasses
$ export OpenBLAS_HOME=../OpenBLAS/

Point the compiler to your shared libraries with the following:

$ export LD_LIBRARY_PATH=~/anaconda3/envs/<your conda env name>/lib/

Note that if you are in your base conda environment, this path will be the simpler ~/anaconda3/lib/ (paths may also change if you use an alternative conda distribution like Miniconda or Miniforge, rather than Anaconda3).

Set the remaining necessary env variables and install PyTorch:

$ export USE_OPENMP=1
$ export USE_MKLDNN=1
$ export USE_MKLDNN_CBLAS=1
$ export BLAS=OpenBLAS
$ export USE_NNPACK=0
$ export USE_XNNPACK=0
$ export USE_QNNPACK=0
$ export USE_PYTORCH_QNNPACK=1
$ python setup.py develop
$ python setup.py install
$ cd ..

Installing and Running InstructLab

Install one last dependency that pip cannot handle by itself:

$ conda install -c rocketce pyarrow

From your main instructlab directory, you should now be able to install the wheel file.

$ pip install git+https://github.com/instructlab/instructlab.git@stable

You should now be able to run InstructLab! The following commands from InstructLab's readme show how to start the chatbot in your terminal:

$ ilab init
$ ilab download
$ ilab chat

$ ilab chat --model-family merlinite
╭────────────────────────────────────────────────────────── system───────────────────────────────────────────────────────────╮
│ Welcome to InstructLab Chat w/ MODELS/MERLINITE-7B-LAB-Q4_K_M.GGUF (type /h for help)                                      │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
>>> If 4 drivers all reach a 4-way stop sign at the same time in the United States, who should go first?         [S][default]
╭──────────────────────────────────────────── models/merlinite-7b-lab-Q4_K_M.gguf────────────────────────────────────────────╮
│ In this particular situation where four drivers arrive simultaneously at a 4-way stop sign in the United States, there are │
│ several methods to determine who should proceed first. These methods include:                                              │
│                                                                                                                            │
│ 1. **Right-of-way rule:** This rule states that the driver on the right has the right-of-way. However, this rule can lead  │
│ to confusion and conflicts when all drivers arrive at the intersection from different directions.                          │
│ 2. **Yield-to-traffic signal rule:** This rule is more common in the United States. It states that the first driver to     │
│ reach the intersection should yield to any other vehicle that may be approaching from a different direction.               │
│ 3. **The "honest" solution:** The most honest solution would be for all drivers to agree to proceed one by one, taking     │
│ turns to continue their journey. This way, no one is left waiting for an extended period of time, and it ensures fairness  │
│ among the drivers.                                                                                                         │
│                                                                                                                            │
│ These methods can help resolve such situations in a more orderly manner. It's important to note that local laws and        │
│ regulations may also come into play when determining right-of-way at intersections.                                        │

ilab chat starts a server in the background, which serves the model via llama_cpp. If you'd like to use a different model than the default merlinite-7b-lab-4Q_K_M.gguf, you must run the serve yourself:

$ ilab download --repository instructlab/granite-7b-lab-GGUF --filename granite-7b-lab-Q4_K_M.gguf
$ ilab serve --model-path models/granite-7b-lab-Q4_K_M.gguf --model-family merlinite &
$ ilab chat -m models/granite-7b-lab-Q4_K_M.gguf --model-family merlinite
╭──────────────────────────────────────────────────────────── system─────────────────────────────────────────────────────────╮
│ Welcome to InstructLab Chat w/ MODELS/GRANITE-7B-LAB-Q4_K_M.GGUF (type /h for help)                                        │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
>>> Write me the SQL to create a table which stores user IDs, job titles, role IDs, and salaries. The user IDs must be unique, and the role IDs should only be the values which exist in another table called ROLES.
╭─────────────────────────────────────────────── models/granite-7b-lab-Q4_K_M.gguf ──────────────────────────────────────────╮
│ ```sql                                                                                                                     │
│ CREATE TABLE users (                                                                                                       │
│   id INT PRIMARY KEY AUTO_INCREMENT,                                                                                       │
│   first_name VARCHAR(50) NOT NULL,                                                                                         │
│   last_name VARCHAR(50) NOT NULL,                                                                                          │
│   email VARCHAR(254) UNIQUE,                                                                                               │
│   role_id INT REFERENCES roles(id) ON DELETE CASCADE,                                                                      │
│   salary DECIMAL(10, 2)                                                                                                    │
| );                                                                                                                         │
│ ```                                                                                                                        │
│ This SQL query creates a table named `users` with the specified columns. The `id` column is defined as the primary key,    │
│ which is automatically incremented and unique for each row. The `first_name`, `last_name`, and `email` columns are required|
| and have a data type of `VARCHAR(50)`. The `role_id` column is a foreign key that references the `roles` table (aliased as |             
│ `roles_table`) and specifies the `on delete cascade` clause, which automatically updates related records in the `users`    |
| table when records are deleted from the `roles_table`. Finally, the `salary` column is a decimal data type with a precision|
| of 10 and a scale of 2, which can store decimal numbers.                                                                   │

Remember to kill the server process when you are done chatting.