You need:
- a system with Linux. Mine is AlmaLinux 9.5 on IBM Power10.
- enough RAM if you want to test bigger LLMs
- enough space for LLMs
Create a user for ollama:
sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama
Download pre-compiled ollama binary from my site:
sudo curl -L -o /usr/local/bin/ollama https://dl.power-devops.com/ollama
Ensure the correct permissions on the binary:
chown ollama:ollama /usr/local/bin/ollama
chmod 0755 /usr/local/bin/ollama
If you work in SELinux-enabled environment, set the correct SELinux context:
chcon -t bin_t /usr/local/bin/ollama
Create ollama.service file for systemd in /etc/systemd/system/ollama.service:
[Unit]
Description=Ollama Service
After=network-online.target
[Service]
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=$PATH"
[Install]
WantedBy=default.target
Reload systemd and start ollama.service
sudo systemctl daemon-reload
sudo systemctl enable --now ollama.service
Run ollama:
If you want to use another LLM, search through the list of available models at https://ollama.com/search or read the documentation on Github https://github.com/ollama/ollama
Re-compiling the ollama binary
If you don't trust downloading binaries from the Internet (my binary is not signed), you can re-compile OLLAMA. You need the source code from the Github repository (see the link above), C++ compiler, Google Go compiler, and CMake.
After downloading the code, change the following lines in readline/term_linux.go:
import (
"syscall"
"unsafe"
"golang.org/x/sys/unix"
)
const (
tcgets = unix.TCGETS
tcsets = unix.TCSETSF
)
Source for the patch was found in the Internet - https://vivapowerpc.eu/20250204-1600_LLMs_on_ppc64le_with_Ollama
Reconfigure the build procedure with CMake:
cmake -B build
Recompile the code with go:
go build
You should get the ollama binary in the repository directory.
Addition about IBM POWER9 systems with accelerators
Naveed from IBM contacted me after I wrote the blog. He patched Ollama code to support POWER9 Nvidia GPUs with CUDA. The code is available on Github - https://github.com/naveedus/ollama-ppc64le/tree/witherspoon. It works faster than the original code if you have GPUs in your IBM Power AC922.
#PowerDataandAI