Automating AIX updates with Ansible
Installing AIX updates has never been easier with Ansible.
Here’s an example of how I used Ansible to automate some recent AIX technology level and service pack updates on my AIX LPARs.
Key to this functionality was the IBM AIX Ansible Galaxy collection, which includes the required AIX modules for Ansible. See links below. The AIX collection was installed on my Ansible controller.
# ansible-galaxy collection list | grep aix
ibm.power_aix 1.8.2
I created several AIX playbooks, which I then combined into a single (main) playbook for execution. Here’s what it looks like:
aix_update_main.yml
---
- name: Display AIX level before update
import_playbook: aix_oslevel_check_quark.yml
- name: Create alternate rootvg on hdisk1
import_playbook: mk_alt_rvg1.yml
- name: Update AIX and Reboot
import_playbook: aix_update_reboot_quark.yml
- name: Display AIX level after update
import_playbook: aix_oslevel_check_quark.yml
The (aix_update) main playbook performs the following steps:
1. Displays the current AIX level, before installing any updates on the AIX host.
2. Creates an alternate rootvg disk on a spare disk, which in this case is hdisk1. This rootvg clone can used for rollback to the previous AIX level if needed.
3. Performs the AIX update. The TL/SP update is installed using NIM (nimclient). When the update is complete, the AIX host is rebooted on the new AIX level.
4. Finally, it displays the AIX level on the AIX host after the updates have been installed and the host restarted.
Before I run the playbook, the AIX hosts is running AIX 7.3 TL0 SP1, rootvg is on hdisk0 and hdisk1 is a spare disk which can be used for cloning the running rootvg. For demonstration purposes, I limited the playbook to run on one AIX host only. The host’s name is quark. The playbook will update quark to AIX 7.3 TL2 SP1.
# ansible -m shell -a 'oslevel -s ; lspv' quark
quark | CHANGED | rc=0 >>
7300-00-01-2148
hdisk0 00f9c18fa8bd126b rootvg active
hdisk1 00f9c18fc724a0ef None
Here’s the contents of each of the playbooks in the main (aix_update) playbook file:
aix_oslevel_check_quark.yml
---
- name: AIX oslevel checking playbook
hosts: quark
tasks:
- name: Display AIX level on AIX host
shell: "oslevel -s"
register: output_oslevel
- name: Print the oslevel
debug:
msg: "{{ ansible_hostname }} has the AIX oslevel of {{ output_oslevel.stdout }}"
mk_alt_rvg1.yml
---
- name: "Create alt rootvg"
hosts: quark
gather_facts: true
vars:
mydisk: hdisk1
collections:
- ibm.power_aix
tasks:
- name: Perform an alternate disk copy of the rootvg to hdisk1 (“alt_root_vg – Create/Cleanup an alternate rootvg disk on a VIOS”)
alt_disk:
action: copy
targets: "{{ mydisk }}"
register: result
- name: Display result
debug: var=result
aix_update_reboot_quark.yml
---
- name: Update AIX server using NIM
gather_facts: false
hosts: quark
vars:
aixver: 7300-02-01-2346
lppsrc: AIX73TL2SP1
reboot: true
tasks:
- name: Check if NIM client is configured
ansible.builtin.command:
cmd: nimclient -l master
changed_when: false
register: registered
ignore_errors: true
- name: Stop if the client is not registered
meta: end_play
when: registered.rc != 0
- name: Get NIM client AIX version
ansible.builtin.command:
cmd: oslevel -s
changed_when: false
register: oslevel
- name: Stop if the client is already updated
meta: end_play
when: oslevel.stdout == aixver
- name: Update AIX on NIM client
ansible.builtin.command:
cmd: nimclient -o cust -a lpp_source={{ lppsrc }} -a fixes=update_all -a accept_licenses=yes
- name: Reboot NIM client/AIX host
ibm.power_aix.reboot:
when: reboot
To execute the playbook, I ran the following command from my Ansible controller:
# ansible-playbook aix_update_main.yml
PLAY [AIX oslevel checking playbook] **************************************************************************************************
TASK [Gathering Facts] ****************************************************************************************************************
ok: [quark]
TASK [Display AIX level on AIX host] **************************************************************************************************
changed: [quark]
TASK [Print the oslevel] **************************************************************************************************************
ok: [quark] => {
"msg": "quark has the AIX oslevel of 7300-00-01-2148"
}
PLAY [Create alt rootvg] **************************************************************************************************************
TASK [Gathering Facts] ****************************************************************************************************************
ok: [quark]
TASK [Perform an alternate disk copy of the rootvg to hdisk1] *************************************************************************
changed: [quark]
TASK [Display result] *****************************************************************************************************************
ok: [quark] => {
"result": {
"changed": true,
"cmd": "alt_disk_copy -d hdisk1 -B",
"failed": false,
"msg": "alt_disk copy operation completed successfully",
"rc": 0,
"stderr": "",
"stderr_lines": [],
"stdout": "Calling mkszfile to create new /image.data file.\nChecking disk sizes.\nCreating cloned rootvg volume group and associated logical volumes.\nCreating logical volume alt_hd5\nCreating logical volume alt_hd6\nCreating logical volume alt_hd8\nCreating logical volume alt_hd4\nCreating logical volume alt_hd2\nCreating logical volume alt_hd9var\nCreating logical volume alt_hd3\nCreating logical volume alt_hd1\nCreating logical volume alt_hd10opt\nCreating logical volume alt_hd11admin\nCreating logical volume alt_lg_dumplv\nCreating logical volume alt_livedump\nCreating /alt_inst/ file system.\nCreating /alt_inst/admin file system.\nCreating /alt_inst/home file system.\nCreating /alt_inst/opt file system.\nCreating /alt_inst/tmp file system.\nCreating /alt_inst/usr file system.\nCreating /alt_inst/var file system.\nCreating /alt_inst/var/adm/ras/livedump file system.\nGenerating a list of files\nfor backup and restore into the alternate file system...\nBacking-up the rootvg files and restoring them to the \nalternate file system...\nModifying ODM on cloned disk.\nBuilding boot image on cloned disk.\nforced unmount of /alt_inst/var/adm/ras/livedump\nforced unmount of /alt_inst/var/adm/ras/livedump\nforced unmount of /alt_inst/var\nforced unmount of /alt_inst/var\nforced unmount of /alt_inst/usr\nforced unmount of /alt_inst/usr\nforced unmount of /alt_inst/tmp\nforced unmount of /alt_inst/tmp\nforced unmount of /alt_inst/opt\nforced unmount of /alt_inst/opt\nforced unmount of /alt_inst/home\nforced unmount of /alt_inst/home\nforced unmount of /alt_inst/admin\nforced unmount of /alt_inst/admin\nforced unmount of /alt_inst\nforced unmount of /alt_inst\nChanging logical volume names in volume group descriptor area.\nFixing LV control blocks...\nFixing file system superblocks...\n",
"stdout_lines": [
"Calling mkszfile to create new /image.data file.",
"Checking disk sizes.",
"Creating cloned rootvg volume group and associated logical volumes.",
"Creating logical volume alt_hd5",
"Creating logical volume alt_hd6",
"Creating logical volume alt_hd8",
"Creating logical volume alt_hd4",
"Creating logical volume alt_hd2",
"Creating logical volume alt_hd9var",
"Creating logical volume alt_hd3",
"Creating logical volume alt_hd1",
"Creating logical volume alt_hd10opt",
"Creating logical volume alt_hd11admin",
"Creating logical volume alt_lg_dumplv",
"Creating logical volume alt_livedump",
"Creating /alt_inst/ file system.",
"Creating /alt_inst/admin file system.",
"Creating /alt_inst/home file system.",
"Creating /alt_inst/opt file system.",
"Creating /alt_inst/tmp file system.",
"Creating /alt_inst/usr file system.",
"Creating /alt_inst/var file system.",
"Creating /alt_inst/var/adm/ras/livedump file system.",
"Generating a list of files",
"for backup and restore into the alternate file system...",
"Backing-up the rootvg files and restoring them to the ",
"alternate file system...",
"Modifying ODM on cloned disk.",
"Building boot image on cloned disk.",
"forced unmount of /alt_inst/var/adm/ras/livedump",
"forced unmount of /alt_inst/var/adm/ras/livedump",
"forced unmount of /alt_inst/var",
"forced unmount of /alt_inst/var",
"forced unmount of /alt_inst/usr",
"forced unmount of /alt_inst/usr",
"forced unmount of /alt_inst/tmp",
"forced unmount of /alt_inst/tmp",
"forced unmount of /alt_inst/opt",
"forced unmount of /alt_inst/opt",
"forced unmount of /alt_inst/home",
"forced unmount of /alt_inst/home",
"forced unmount of /alt_inst/admin",
"forced unmount of /alt_inst/admin",
"forced unmount of /alt_inst",
"forced unmount of /alt_inst",
"Changing logical volume names in volume group descriptor area.",
"Fixing LV control blocks...",
"Fixing file system superblocks..."
]
}
}
PLAY [Update AIX server using NIM] ****************************************************************************************************
TASK [Check if NIM client is configured] **********************************************************************************************
ok: [quark]
TASK [Stop if the client is not registered] *******************************************************************************************
skipping: [quark]
TASK [Get NIM client AIX version] *****************************************************************************************************
ok: [quark]
TASK [Stop if the client is already updated] ******************************************************************************************
skipping: [quark]
TASK [Update AIX on NIM client] *******************************************************************************************************
changed: [quark]
TASK [Reboot NIM client/AIX host] *****************************************************************************************************
changed: [quark]
PLAY [AIX oslevel checking playbook] **************************************************************************************************
TASK [Gathering Facts] ****************************************************************************************************************
ok: [quark]
TASK [Display AIX level on AIX host] **************************************************************************************************
changed: [quark]
TASK [Print the oslevel] **************************************************************************************************************
ok: [quark] => {
"msg": "quark has the AIX oslevel of 7300-02-01-2346"
}
PLAY RECAP ****************************************************************************************************************************
quark : ok=13 changed=5 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
After the playbook has finished running, and quark rebooted, the AIX level is now 7300-02-01-2346 and hdisk1 is a clone of rootvg before the update was installed (and can be used to return to the previous AIX level if required).
# ansible -m shell -a 'oslevel -s ; lspv' quark
quark | CHANGED | rc=0 >>
7300-02-01-2346
hdisk0 00f9c18fa8bd126b rootvg active
hdisk1 00f9c18fc724a0ef altinst_rootvg
If I need to roll back to the previous TL/SP, I can run another playbook to achieve this for me.
rollback_main.yml
- name: Change bootlist to hdisk1 for rollback
import_playbook: setbootlist1.yml
- name: Display bootlist
import_playbook: getbootlist.yml
- name: Reboot from old rootvg
import_playbook: aix_reboot_quark.yml
- name: Display AIX level after reboot
import_playbook: aix_oslevel_check_quark.yml
The (rollback) main playbook performs the following steps:
1. Changes the bootlist to hdisk1, which is the alternate rootvg at the previous AIX level.
2. Displays the bootlist. Checking that the bootlist has changed to hdisk1.
3. Reboots the AIX host from the alternate rootvg (the previous AIX level).
4. Displays the AIX level after the reboot. Checking that the previous AIX level is in use now.
Here’s the contents of each of the playbooks in the main (rollback) playbook file:
setbootlist1.yml
---
- name: "Change bootlist"
hosts: quark
gather_facts: true
vars:
mydisk: hdisk1
collections:
- ibm.power_aix
tasks:
- name: Set normal boot list to hdisk1
bootlist:
normal:
- device: "{{ mydisk }}"
register: result
- name: Display result
debug: var=result
getbootlist.yml
---
- name: "Display bootlist"
hosts: quark
gather_facts: true
collections:
- ibm.power_aix
tasks:
- name: Retrieve normal boot list
bootlist:
- name: Print the boot list
debug:
var: ansible_facts.bootlist
aix_reboot_quark.yml
---
- name: AIX reboot playbook
gather_facts: false
hosts: quark
vars:
reboot: true
tasks:
- name: Reboot AIX host
ibm.power_aix.reboot:
when: reboot
register: result
- name: Display result
debug: var=result
aix_oslevel_check_quark.yml
---
- name: AIX oslevel checking playbook
hosts: quark
tasks:
- name: Display AIX level on AIX host
shell: "oslevel -s"
register: output_oslevel
- name: Print the oslevel
debug:
msg: "{{ ansible_hostname }} has the AIX oslevel of {{ output_oslevel.stdout }}"
Here’s the output from the rollback playbook:
# ansible-playbook rollback_main.yml
PLAY [Change bootlist] ****************************************************************************************************************
TASK [Gathering Facts] ****************************************************************************************************************
ok: [quark]
TASK [Set normal boot list to hdisk1] *************************************************************************************************
changed: [quark]
TASK [Display result] *****************************************************************************************************************
ok: [quark] => {
"result": {
"ansible_facts": {
"bootlist": {
"normal": [
{
"blv": "hd5",
"device": "hdisk1",
"pathid": "0"
}
],
"service": [
{
"device": "cd0"
},
{
"device": "hdisk0",
"pathid": "0"
},
{
"bserver": "000.000.000.000",
"client": "000.000.000.000",
"device": "ent0",
"duplex": "auto",
"gateway": "000.000.000.000",
"speed": "auto",
"vlan_tag": "5"
}
]
}
},
"changed": true,
"failed": false,
"msg": "",
"stderr": "",
"stderr_lines": [],
"stdout": "'ibm,max-boot-devices' = 0x5\nmatch_specific_info: ut=disk/vscsi/cvdisk\nbootinfo_aix = /vdevice/v-scsi@30000004/disk@8300000000000000:1\nboot list for device: '/vdevice/v-scsi@30000004/disk@8300000000000000:2'\nSetting NVRAM variable: (ibm,aix-fc-bootinfo=)\nSetting NVRAM variable: (boot-file=)\nSetting NVRAM variable: (boot-device=)\nSetting NVRAM variable: (bootinfo-aix=)\nSetting NVRAM variable: (bootinfo-aix=/vdevice/v-scsi@30000004/disk@8300000000000000:1)\nSetting NVRAM variable: (boot-command=boot)\nSetting NVRAM variable: (boot-device=)\nSetting NVRAM variable: (boot-device=/vdevice/v-scsi@30000004/disk@8300000000000000:2)\n",
"stdout_lines": [
"'ibm,max-boot-devices' = 0x5",
"match_specific_info: ut=disk/vscsi/cvdisk",
"bootinfo_aix = /vdevice/v-scsi@30000004/disk@8300000000000000:1",
"boot list for device: '/vdevice/v-scsi@30000004/disk@8300000000000000:2'",
"Setting NVRAM variable: (ibm,aix-fc-bootinfo=)",
"Setting NVRAM variable: (boot-file=)",
"Setting NVRAM variable: (boot-device=)",
"Setting NVRAM variable: (bootinfo-aix=)",
"Setting NVRAM variable: (bootinfo-aix=/vdevice/v-scsi@30000004/disk@8300000000000000:1)",
"Setting NVRAM variable: (boot-command=boot)",
"Setting NVRAM variable: (boot-device=)",
"Setting NVRAM variable: (boot-device=/vdevice/v-scsi@30000004/disk@8300000000000000:2)"
]
}
}
PLAY [Display bootlist] ***************************************************************************************************************
TASK [Gathering Facts] ****************************************************************************************************************
ok: [quark]
TASK [Retrieve normal boot list] ******************************************************************************************************
ok: [quark]
TASK [Print the boot list] ************************************************************************************************************
ok: [quark] => {
"ansible_facts.bootlist": {
"normal": [
{
"blv": "hd5",
"device": "hdisk1",
"pathid": "0"
}
],
"service": [
{
"device": "cd0"
},
{
"device": "hdisk0",
"pathid": "0"
},
{
"bserver": "000.000.000.000",
"client": "000.000.000.000",
"device": "ent0",
"duplex": "auto",
"gateway": "000.000.000.000",
"speed": "auto",
"vlan_tag": "5"
}
]
}
}
PLAY [AIX reboot playbook] ************************************************************************************************************
TASK [Reboot AIX host] ****************************************************************************************************************
changed: [quark]
TASK [Display result] *****************************************************************************************************************
ok: [quark] => {
"result": {
"changed": true,
"elapsed": "184 sec",
"failed": false,
"msg": "System has been rebooted SUCCESSFULLY",
"rebooted": true
}
}
PLAY [AIX oslevel checking playbook] **************************************************************************************************
TASK [Gathering Facts] ****************************************************************************************************************
ok: [quark]
TASK [Display AIX level on AIX host] **************************************************************************************************
changed: [quark]
TASK [Print the oslevel] **************************************************************************************************************
ok: [quark] => {
"msg": "quark has the AIX oslevel of 7300-00-01-2148"
}
PLAY RECAP ****************************************************************************************************************************
quark : ok=11 changed=3 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
After the playbook has finished running, and quark rebooted, the AIX level is now the previous level 7300-00-01-2148 and hdisk1 is now the running rootvg. In this situation, when I’m ready to attempt the TL/SP update again, I have another set of playbooks that will prepare the host, again, for the next update. But I’ll cover this in a future post.
# ansible -m shell -a 'oslevel -s ; lspv' quark
quark | CHANGED | rc=0 >>
7300-00-01-2148
hdisk0 00f9c18fa8bd126b old_rootvg
hdisk1 00f9c18fc724a0ef rootvg active
Note: For the AIX reboot function to work as expected, we added the following ssh_args entry to our /etc/ansible/ansible.cfg file, under the [defaults] section (without this the reboot can hang or the playbook return an unexpected error):
[defaults]
# (boolean) By default Ansible will issue a warning when received from a task action (module or action plugin)
# These warnings can be silenced by adjusting this setting to False.
;action_warnings=True
host_key_checking = False
#interpreter_python = /usr/bin/python3
interpreter_python = /opt/freeware/bin/python3
[ssh_connection]
ssh_args = -o ForwardAgent=yes -o ControlPersist=30m -o ServerAliveInterval=45 -o ServerAliveCountMax=10
Of course, there’s always more than one way to skin a catfish, and I’m sure others have their own preferred approach to using Ansible for this type of task. Here’s one, for example, https://www.ansible.com/blog/aix-patch-management-with-ansible/. However, I hope this gives you some idea as to how you might be able to employ Ansible to help you manage your AIX TL/SP updates with automation.
Want to learn more about Ansible on Power/AIX? Consider attending the new Ansible on IBM Power Workshop training course. It’ll get you up to speed quickly with Ansible on Power.
Ansible on IBM Power Workshop
https://www.ibm.com/training/course/QZC51G
Refer to the Redbooks publication, Using Ansible for Automation in IBM Power Environments, for a wealth of information on using Ansible in Power server environments. This book is full of great, practical, technical information for Power administrators.
Using Ansible for Automation in IBM Power Environments
https://www.redbooks.ibm.com/redpieces/abstracts/sg248551.html
References
https://ibm.github.io/ansible-power-aix/modules/alt_disk.html
https://community.ibm.com/community/user/power/discussion/share-your-thoughts-on-aix-automation-use-cases-of-high-value