Hi all,
in the last couple of days, we experienced issues with the AIX Toolbox on several different AIX 7.3 systems. Some of them are at 7300-03-01, others at 7300-02-03.
All of them are pretty different and affect different customers of ours, but all of them have similar symptoms and issues with DNF.
In the test environment of my company, we experienced this issue on a NIM server. It has never seen any other RPM package apart from those from the AIX toolbox. We keep this server quite up to date in terms of AIX, fixes and RPM packages.
A couple of days ago, I did a "dnf update", and it messed up DNF itself completely, leaving the whole DNF completely unusable with broken python dependencies. As I did not find any other solution, I decided to do a complete reinstall of all RPM packages; so I purged every single RPM package, removed /opt/freeware/libs, reinstalled the rpm.rte LPP-package, reinstalled DNF and reinstalled all the packages that were installed before. One of those packages was ansible. While DNF was working (a bit, at least, see below), Ansible is not:
ansibleadm@nimserver:/ansible $ ansible-playbook -v update_all_systems.yml
Using /etc/ansible/ansible.cfg as config file
[ERROR]: Unexpected Exception, this is probably a bug: function() argument 'code' must be code, not str
Traceback (most recent call last):
File "/opt/freeware/lib/python3.12/site-packages/ansible/cli/__init__.py", line 660, in cli_executor
exit_code = cli.run()
^^^^^^^^^
File "/opt/freeware/lib/python3.12/site-packages/ansible/cli/playbook.py", line 153, in run
results = pbex.run()
^^^^^^^^^^
File "/opt/freeware/lib/python3.12/site-packages/ansible/executor/playbook_executor.py", line 117, in run
self._tqm.load_callbacks()
File "/opt/freeware/lib/python3.12/site-packages/ansible/executor/task_queue_manager.py", line 206, in load_callbacks
stdout_callback = callback_loader.get(self._stdout_callback_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/freeware/lib/python3.12/site-packages/ansible/plugins/loader.py", line 979, in get
ctx = self.get_with_context(name, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/freeware/lib/python3.12/site-packages/ansible/plugins/loader.py", line 1024, in get_with_context
self._module_cache[path] = self._load_module_source(python_module_name=plugin_load_context._python_module_name, path=path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/freeware/lib/python3.12/site-packages/ansible/plugins/loader.py", line 943, in _load_module_source
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/ansible/.ansible/collections/ansible_collections/community/general/plugins/callback/yaml.py", line 46, in <module>
class MyDumper(AnsibleDumper):
TypeError: function() argument 'code' must be code, not str
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/freeware/lib/python3.12/site-packages/ansible/cli/__init__.py", line 669, in cli_executor
raise AnsibleError("Unexpected Exception, this is probably a bug.") from ex
ansible.errors.AnsibleError: Unexpected Exception, this is probably a bug: function() argument 'code' must be code, not str
When trying to do another update via DNF, I get this:
# dnf update
Last metadata expiration check: 6:24:11 ago on November 20, 2025, 08:08:49 AM CET.
Error:
Problem: package python3.9-3.9.24-1.ppc from ppc requires /opt/freeware/libexec/python3.9_32, but none of the providers can be installed
- cannot install both python3.9-3.9.24-1.ppc from ppc and python3.9-3.9.16-0.ppc from @System
- cannot install the best update candidate for package python3.9-3.9.16-0.ppc
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
This has been at least the 4th time on this server within the last 2 years that I have to completely reinstall DNF and all packages (and with it, all configurations). And all I need to do to mess up DNF is "dnf update".
As mentioned earlier, we experienced similar situations at customers like a dozen times in the last two years, and 2 more occurances of this within the last 3 days.
What is going on here? How can we make the AIX Toolbox work reliably? What is your update strategy to prevent things like this? Is the RPM repository for AIX Toolbox really inconsistent? If so, does it really happen approx. twice a year?
If any more information is needed, please let me know.
Best regards,
Thomas