Hey all,
I want to share my experience with AI on AIX and would like to know if you have already run some tests of AI on AIX. Which models you have been working with and what use cases do you have identified?
Our test case was triggered by Db2 and the following Github project: https://github.com/shaikhq/db2-langchain-rag-local
Downloads
- llama: https://github.com/ggml-org/llama.cpp
- qwen model: https://huggingface.co/Qwen/Qwen2.5-3B-Instruct-GGUF/resolve/main/qwen2.5-3b-instruct-q4_k_m.gguf
Prepare AIX
dnf install gcc gcc-c++ cmake python3-numpy-2.3.2 python3.12-pyyaml-6.0.2
# tqdm is not available yet, but I was able to copy the files from MacOS into /opt/freeware/lib64/python3.12/site-packages/tqdm
Build llama (copy the downloaded llama from Github to your AIX test system and extract the zip file if needed)
# Set all the required vars
export PATH=/opt/freeware/bin:$PATH
export LESS=eFRX
ulimit -m unlimited
ulimit -d unlimited
ulimit -f unlimited
ulimit -c unlimited
export OBJECT_MODE=64
export CC="gcc"
export CXX="g++"
export CFLAGS="-pthread -maix64 -lm"
export CXXFLAGS="-pthread -maix64 -lm"
export LDFLAGS="-lm -lbsd -lcrypt -DLLAMA_NATIVE=ON -DLLAMA_BUILD_SERVER=ON -DBUILD_SHARED_LIBS=ON"
# Move into the llama directory and run cmake
cd llama.cpp
cmake -B build_llama -DLLAMA_CURL=OFF
cmake --build build_llama/
Prepare the model from huggingface for big endian (copy the downloaded Model from Huggingface to your AIX test system)
python3.12 gguf-py/gguf/scripts/gguf_convert_endian.py /your/path/qwen2.5-3b-instruct.Q4_K_M.gguf native
Run the model
# ./build_llama/bin/llama-simple-chat -m /your/path/qwen2.5-3b-instruct.Q4_K_M.gguf
........................................................................................
> Hello world!
Hello! It's nice to meet you. Is there something you would like to talk about or ask? I'm here to help with any information you need.
Many thanks to our contacts at IBM for helping us with this!
Best regards
------------------------------
Niklas Vosskoetter
Deputy Team Lead & System Engineer Unix and Cloud Service Power
------------------------------