You need:

• a system with Linux. Mine is AlmaLinux 9.5 on IBM Power10.
• enough RAM if you want to test bigger LLMs
• enough space for LLMs

Create a user for ollama:

sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama

Download pre-compiled ollama binary from my site:

sudo curl -L -o /usr/local/bin/ollama https://dl.power-devops.com/ollama

Ensure the correct permissions on the binary:

chown ollama:ollama /usr/local/bin/ollama
chmod 0755 /usr/local/bin/ollama

If you work in SELinux-enabled environment, set the correct SELinux context:

chcon -t bin_t /usr/local/bin/ollama

Create ollama.service file for systemd in /etc/systemd/system/ollama.service:

[Unit]
Description=Ollama Service
After=network-online.target

[Service]
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=$PATH"

[Install]
WantedBy=default.target

Reload systemd and start ollama.service

sudo systemctl daemon-reload
sudo systemctl enable --now ollama.service

Run ollama:

If you want to use another LLM, search through the list of available models at https://ollama.com/search or read the documentation on Github https://github.com/ollama/ollama

Re-compiling the ollama binary

If you don't trust downloading binaries from the Internet (my binary is not signed), you can re-compile OLLAMA. You need the source code from the Github repository (see the link above), C++ compiler, Google Go compiler, and CMake.

After downloading the code, change the following lines in readline/term_linux.go:

import (
        "syscall"
        "unsafe"

        "golang.org/x/sys/unix"
)

const (
        tcgets = unix.TCGETS
        tcsets = unix.TCSETSF
)

Source for the patch was found in the Internet - https://vivapowerpc.eu/20250204-1600_LLMs_on_ppc64le_with_Ollama

Reconfigure the build procedure with CMake:

cmake -B build

Recompile the code with go:

go build

You should get the ollama binary in the repository directory.

Addition about IBM POWER9 systems with accelerators

Naveed from IBM contacted me after I wrote the blog. He patched Ollama code to support POWER9 Nvidia GPUs with CUDA. The code is available on Github - https://github.com/naveedus/ollama-ppc64le/tree/witherspoon. It works faster than the original code if you have GPUs in your IBM Power AC922.