Introduction
Amidst the growing AI hype, IBM Power is positioning itself as a leading inference platform, offering efficient processing capabilities to meet the demands of AI workloads and real-time data analysis. Ansible is the perfect choice to support setting up IBM Power environments for various AI use cases.
Ansible is an open-source automation tool that simplifies the management and configuration of IT infrastructure. It allows system administrators and DevOps teams to automate tasks such as application deployment, configuration management, and continuous integration. Known for its simplicity and scalability, Ansible uses a declarative language to define automation tasks, making it both powerful and user-friendly. This primer will guide you through the basics of Ansible, helping you understand how it works and how to get started with automating your infrastructure.
Why am I using Ansible?
As part of my daily work, I‘m configuring and deploying components on IBM Power systems, creating assets for our clients, and porting software to the ppc64le-architecture. These activities include repetitive tasks and require reproducible results. Instead of relying on unstable shell scripts or opening SSH sessions to manually configure the target systems, I build my workflow based on the declarative nature of configurable Ansible playbooks.
Fortunately for all people not working regularly with these technologies, the Ansible playbooks are shared online, so you don't have to build everything from scratch on your own. Before reusing existing works, let have a look at the core components of Ansible and understand what we are using.
Core components
1. Inventory
The inventory is a YAML (Yet Another Markup Language) or INI-file that contains information about the managed hosts (servers or machines) and their details (e.g., IP addresses or hostnames, login credentials). It defines the target machines where Ansible will apply configurations or run tasks. The inventory can be static (a simple text file) or dynamic (generated by scripts, particularly useful in cloud environments).
Here is a basic example of an inventory file:
[web_servers]
server1.example.com
server2.example.com
[db_servers]
db1.example.com
It defines two host groups web_servers and db_servers. Thus, certain actions can be applied to web servers, db servers, or both.
An inventory can include much more information like variable definitions to manipulate the execution flow of playbooks. Here is a more complex inventory file, which defines a llm_servers and a db_servers group with custom variable definitions:
llm_servers:
hosts:
llm_host_1:
ansible_host: 0.0.0.1
ansible_password: "secure password 01"
llm_host_2:
ansible_host: 0.0.0.2
ansible_password: "secure password 02"
vars:
ansible_user: technical_llm_user
conda_dir: "/home/{{ ansible_user }}/micromamba"
micromamba_location: "/usr/local/bin/micromamba"
model_repository: ibm-granite/granite-3.2-8b-instruct
python_version: 3.11
working_directory: "/home/{{ ansible_user }}/llm"
llama_cpp_args:
v:
c: 12288
t: 128
tb: 128
llama_cpp_argv:
api-key: examplekey001
host: 0.0.0.0
port: 8080
db_servers:
hosts:
db_host_1:
ansible_host: 0.0.0.5
ansible_password: "secure password 05"
vars:
ansible_user: technical_db_user
enable_pgvector: true
2. Playbooks
Playbooks are YAML files that define a series of plays that Ansible will perform on a group of hosts. They describe the automation process in a human-readable way and are the heart of Ansible automation. They can define multiple plays, which consist of one or more tasks. Furthermore, plays can have variable definitions, conditions, and loops, making them very powerful.
In the following example, one play for installing Apache web servers on a list of managed hosts (web_servers) is defined. It consists of a single task, which uses the builtin dnf module to install the apache2 package on a Red Hat Enterprise Linux (RHEL)-based system.
- name: Install Apache on web servers
hosts: web_servers
become: yes
tasks:
- name: Install Apache
ansible.builtin.dnf:
name: apache2
state: present
3. Modules
Modules are the core units of work in Ansible. They are built-in or custom scripts that perform specific tasks, such as installing packages, copying files, or managing services.
Modules abstract complex operations into simple commands, making it easy to automate tasks without needing to write shell scripts or code.
For example, the built-in dnf module installs a package on a RHEL-based system:
- name: Install Apache
ansible.builtin.dnf:
name: apache2
state: present
4. Tasks
Tasks are individual actions within a playbook. Each task runs a module to perform a specific operation on the managed hosts. Tasks are the building blocks of a playbook, and they describe what action Ansible will take on the target hosts. A task calls a certain module like dnf for installing packages on a RHEL system or copy for copying files from your host to the target system.
5. Variables
Variables are a great way to store values that can be reused throughout your playbooks or roles. They allow you to make your playbooks dynamic and use different configurations depending on the environment or other conditions.
Below, you will see an example of using a variable holding a version number, which is used to get the correct RPM-file (Red Hat Package Manager).
vars:
rhel_version: 9
tasks:
- rpm_key:
state: present
key: "https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-{{ rhel_version }}"
become: true
become_user: root
- name: Install enterprise package
become: true
become_user: root
ansible.builtin.dnf:
name:
- dnf-plugins-core
- "https://dl.fedoraproject.org/pub/epel/epel-release-latest-{{ rhel_version }}.noarch.rpm"
state: latest
Instead of statically assigning a variable value, you can dynamically determine a variable‘s value and use it later. Here, register is used to capture the result from the shell module and store it in a variable called rhel_version:
tasks:
- name: Get RHEL version
ansible.builtin.shell: rpm -E %rhel
register: rhel_version
- name: Print RHEL version
ansible.builtin.debug:
msg: "RHEL version: {{ rhel_version.stdout }}"
- rpm_key:
state: present
key: "https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-{{ rhel_version.stdout }}"
become: true
become_user: root
- name: Install enterprise package
become: true
become_user: root
ansible.builtin.dnf:
name:
- dnf-plugins-core
- "https://dl.fedoraproject.org/pub/epel/epel-release-latest-{{ rhel_version.stdout }}.noarch.rpm"
state: latest
Sometimes, it is helpful to set a variable to a different value for a specific run to manipulate the execution flow, e.g. to disable a play installing a database or disable running smoke tests a the end.
There are two ways to pass variables to playbook executions. The first option is to pass them as simple key-value pairs. Notice, that Ansible will automatically treat all values as strings:
$ ansible-playbook -i inventory.yml --extra-vars "backend=vllm python_version=3.11" my_playbook.yml
For more complex and data type-agnostic situations, you can pass a JSON-string to the --extra-vars argument:
$ ansible-playbook -i inventory.yml --extra-vars '{"python_version": "3.11", "run_smoke_test": false, "rhel_version": 9}' my_playbook.yml
6. Roles
Roles are a way to organize and reuse Ansible playbooks. They group related tasks, variables, and files into a structured directory format. This makes it easier to maintain and share Ansible automation. Furthermore, they allow you to reuse common configurations, reducing duplication.
A role might include tasks for installing a runtime for serving large language models, configuration files, and chat templates for your models.
Other useful components
Ansible offers further components giving you more flexible. This includes templates and handlers. The latter are triggered when specific events occur. Additionally, there is the community-driven Ansible Galaxy platform, where people can share roles and collections.
Writing Your First Ansible Playbook
The easiest way to get started with Ansible, is to install it via Python‘s pip package manager:
$ python -m pip install ansible
Note: You may want to create a virtual environment before installing Ansible, so you don‘t mess up your system-wide environment.
For a basic example, we need three files: An inventory to define the hosts we want to manage, a playbook to define the actions we want to perform on the target systems, and a little Python script that we use to demonstrate how to copy files to a target system. Let's create these files via command-line in our working directory:
$ touch inventory.yml my_playbook.yml main.py
We use a very simple inventory to configure a single LPAR:
techzone:
hosts:
my_host_1:
ansible_host: <INSERT IP ADDRESS HERE>
ansible_password: "<INSERT PASSWORD HERE>"
vars:
ansible_user: <INSERT USERNAMEHERE>
working_directory: "/home/{{ ansible_user }}/llm"
copy_script: true
In our playbook, we implement a single play, which is applied to all hosts associated with the techzone group. We create nine tasks installing basic dependencies, create a working directory on the target system and copy our Python script to it.
- name: Set up system
hosts: techzone
tasks:
- name: Ping my host
ansible.builtin.ping:
- name: Get RHEL version
ansible.builtin.shell: rpm -E %rhel
register: rhel_version
- name: Print RHEL version
ansible.builtin.debug:
msg: "RHEL version: {{ rhel_version.stdout }}"
- rpm_key:
state: present
key: "https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-{{ rhel_version.stdout }}"
become: true
become_user: root
- name: Install enterprise package
become: true
become_user: root
dnf:
name:
- dnf-plugins-core
- "https://dl.fedoraproject.org/pub/epel/epel-release-latest-{{ rhel_version.stdout }}.noarch.rpm"
state: latest
- name: Enable powertools/codeready builder for RHEL (powertools under CentOS)
become: true
become_user: root
command: subscription-manager repos --enable codeready-builder-for-rhel-{{ rhel_version.stdout }}-ppc64le-rpms
- name: Install the 'Development tools' package group
become: true
become_user: root
dnf:
name: '@Development tools'
state: present
- name: Create working directory
ansible.builtin.file:
path: "{{ working_directory }}"
state: directory
owner: "{{ ansible_user }}"
mode: 0775
recurse: yes
- name: Copy sample Python script
ansible.builtin.copy:
src: main.py
dest: "{{ working_directory }}/main.py"
owner: "{{ ansible_user }}"
mode: u=rw,g=r,o=rwx
when: copy_script is defined and copy_script
Notice, that the last task (the copy-task) has a conditional that needs to be met in order to execute the task: The variable copy_script needs to be defined and set to true if it exists. You can learn more about conditionals here.
become and become_user indicate, which user the Ansible agent should become before executing a certain task. This is useful if you need root permissions for installing certain packages or modifying files in restricted parts of your system.
You may have noticed that values are sometimes wrapped in single- or double-quotes, and sometimes not. As Ansible playbooks are YAML files, all values that include characters with a specific purpose in YAML (like the colon) need to be wrapped in quotes. Additionally, if you want to use variables, they need to be wrapped in quotes, too.
If you want to learn more about the available modules or specific constructs, check out Ansible's online documentation. Each built-in Ansible module has a fantastic documentation, where you find examples and parametrisation options, so don't hesitate using your search engine of choice as you will definitly find good documentation and valuable tips in the online community.
Let's finish this little example by executing the playbook:
$ ansible-playbook -i inventory.yml my_playbook.yml
If the playbook was executed successfully, you should find all the packages we specified installed on your target system and a new directory with the empty Python script in it.
Useful tips and links
Starting at a specific task
Let‘s assume your playbook failed at a certain task at the near end of the play. Instead of running the whole playbook again, you can pick up the work at a specific task by passing the --start-at-task parameter:
$ ansible-playbook -i inventory.yml --start-at-task="Copy sample Python script" my_playbook.yml
Modularity
Ansible allows you to import a file with tasks or plays. This way, you can put tasks you do multiple times in your code to a separate file and import it whenever you need to execute them again. Furthermore, importing a whole playbook gives you even better setup options. Let's say you want the user to decide, which model runtime or database to set up. Depending on the user's choice, the playbook for MariaDB or PostgreSQL is loaded and executed.
In the following snippet, the import_tasks module is used to import tasks from the micromamba-installation.yml file. It is like directly writing the tasks at that place in the playbook. Similarly, import_playbook is used to import the contents of an entire playbook. Here, the postgresql-setup.yml is imported if postgres_setup is defined and set to true.
- name: System setup
hosts: llm_servers
tasks:
- name: Ping my host
ansible.builtin.ping:
- name: Create working directory
ansible.builtin.file:
path: "{{ working_directory }}"
state: directory
owner: "{{ ansible_user }}"
mode: 0775
recurse: yes
- name: Install micromamba
ansible.builtin.import_tasks: micromamba-installation.yml
- name: Import postgresql setup playbook
ansible.builtin.import_playbook: postgresql-setup.yml
when: postgres_setup is defined and postgres_setup
Useful playbooks for AI on IBM Power
You can find a collection of useful Ansible playbooks, which help you setting up AI on IBM Power use cases on your AI on Power DACH organisation on GitHub: AI-on-Power-DACH/ai-on-power-playbooks
Reuse playbooks by downloading them from central repositories
A little trick I use to avoid writing the same plays over and over again is to have a separate preparation playbook that downloads all playbooks I want to reuse to my host machine.
You don‘t need an inventory for this, just a playbook with a single task using the get_url module to download the necessary playbooks. In the following example, we create a notebook called local-setup.yml in our working directory, which looks like this now:
$ tree .
.
└── local-setup.yml
1 directory, 1 file
Inside the playbook, we create a single play, which is applied to localhost, our host machine. We use a loop-construct in our only task to iterate over the list of playbooks that we want to download via the get_url module.
- name: Get all playbooks and tasks
hosts: localhost
tasks:
- name: Download files
ansible.builtin.get_url:
url: "{{ item }}"
dest: .
mode: 0775
loop:
- "https://raw.githubusercontent.com/AI-on-Power-DACH/ai-on-ibm-power-playbooks/refs/heads/main/playbooks/template.service"
- "https://raw.githubusercontent.com/AI-on-Power-DACH/ai-on-ibm-power-playbooks/refs/heads/main/playbooks/pgvector.yml"
- "https://raw.githubusercontent.com/AI-on-Power-DACH/ai-on-ibm-power-playbooks/refs/heads/main/playbooks/postgresql-setup.yml"
After executing the playbook via ...
$ ansible-playbook local-setup.yml
... the three files were downloaded and are now available in your working directory:
$ tree .
.
├── local-setup.yml
├── pgvector.yml
├── postgresql-setup.yml
└── template.service
1 directory, 4 files
Conclusion
In conclusion, Ansible is a powerful and flexible automation tool that simplifies the management of IT infrastructure. By leveraging its easy-to-understand syntax, key components like playbooks, modules, and roles, and its agentless design, Ansible enables users to automate repetitive tasks, improve consistency, and enhance efficiency. Whether you’re just getting started or looking to streamline complex workflows, Ansible offers the tools you need to automate with confidence. As you continue to explore its capabilities, you’ll find that Ansible’s scalability and community-driven resources make it an invaluable asset for any IT professional.