Circling back to this issue.
I tracked down where the jinja2 (AWX) package was located -- /var/lib/awx/venv/awx/lib/python3.7/site-packages -- and went ahead with resetting the (test) AWX environment by re-building from scratch allowing python3.9, ansible-core, and the like to get installed.
I then renamed the existing (AWX) jinja2 directory(s) before copying in a jinja 3.1.3 directory structure, stopped/started AWX, and successfully connected. So far so good! However, when running a Job Template I still hit the issue whereby AWX "thinks" it's still got jinja2 2.10.1 installed so conclude there must be some "master list" of (AWX) packages which are referred to.
If anyone know where this list might be or, in fact, whether there's (even) more needing to copied in, jinja2 wise, I'd be very interested to hear.
Original Message:
Sent: Thu September 28, 2023 08:15 AM
From: Steve Munday
Subject: Ansible feature requests
Andrey,
Hi, understood, a complex issue to resolve.
I'm wondering if a forced downgrade of ansible (so removing ansible-core etc.) would return things to a more stable situation. I think that in doing this, though, would see loads of packages removed etc. etc.
Many thanks, Steve
------------------------------
Steve Munday
AIX, IBM i, HMC, PowerVM
Original Message:
Sent: Thu September 28, 2023 07:49 AM
From: Andrey Klyachkin
Subject: Ansible feature requests
Steve,
I can't test it. But I don't think it'll work.
I am still by rebuilding of AWX. It requires a lot of time and lot of patches. The newest versions of AWX require some packages which are not available for AIX. The older version of AWX, which you have, don't work very well with Python 3.9 and some other newer versions of Python packages. I can't give you any timeline, sorry.
------------------------------
Andrey Klyachkin
https://www.power-devops.com
Original Message:
Sent: Thu September 28, 2023 07:41 AM
From: Steve Munday
Subject: Ansible feature requests
Andrey,
Hi, just a thought, can we overcome the jinja2-3.0.3 issue by doing the following?
Copy the contents of
/opt/freeware/lib/python3.9/site-packages/jinja2
Over the top of
/var/lib/awx/venv/ansible/lib/python3.7/site-packages/jinja2
Or am I over simplifying things too much?
Many thanks, Steve
------------------------------
Steve Munday
AIX, IBM i, HMC, PowerVM
Original Message:
Sent: Mon September 25, 2023 09:57 AM
From: Steve Munday
Subject: Ansible feature requests
Andrey,
Hi, good afternoon.
I was wondering if you had a timeline for the re-packaging of AWX so it uses Python3.9 so that this Jinja2 3.0.3 issue can be resolved?
Many thanks, Steve
------------------------------
Steve Munday
AIX, IBM i, HMC, PowerVM
Original Message:
Sent: Sat September 09, 2023 03:58 PM
From: Andrey Klyachkin
Subject: Ansible feature requests
Hi Steve,
yes, it looks like AWX must be repackaged with new Python 3.9 too. I am not sure that I will be able to do it next week.
------------------------------
Andrey Klyachkin
https://www.power-devops.com
Original Message:
Sent: Fri September 08, 2023 11:40 AM
From: Steve Munday
Subject: Ansible feature requests
Andrey,
Hi, any thoughts on this please? I can't see where AWX is finding Jinja2 2.10 when rpm -qa has 3.0.3 clearly listed.
Many thanks, Steve
------------------------------
Steve Munday
AIX, IBM i, HMC, PowerVM
Original Message:
Sent: Wed September 06, 2023 04:31 AM
From: Steve Munday
Subject: Ansible feature requests
Andrey,
Hi, good morning.
Updating supervisor was successful.
Downloading Packages:Running transaction checkTransaction check succeeded.Running transaction testTransaction test succeeded.Running transaction Preparing : 1/1 Upgrading : supervisor-4.2.0-3aix.noarch 1/2 Running scriptlet: supervisor-4.2.0-3aix.noarch 1/2ln: failed to create symbolic link '/etc/rc.d/rc2.d/Ssupervisord': Do not specify an existing file.ln: failed to create symbolic link '/etc/rc.d/rc2.d/Ksupervisord': Do not specify an existing file.warning: %post(supervisor-4.2.0-3aix.noarch) scriptlet failed, exit status 1Error in POSTIN scriptlet in rpm package supervisor Running scriptlet: supervisor-4.2.0-2aix.noarch 2/2 Cleanup : supervisor-4.2.0-2aix.noarch 2/2 Verifying : supervisor-4.2.0-3aix.noarch 1/2 Verifying : supervisor-4.2.0-2aix.noarch 2/2Upgraded: supervisor-4.2.0-3aix.noarchComplete!#
No more below errors :-)
Problem: package supervisor-4.2.0-2aix.noarch requires /opt/freeware/bin/python3_64, but none of the providers can be installed
I then ran dnf update which was successful. Finally I stopped and started AWX and got the below during the restart.
# /etc/rc.d/init.d/postgresql start;/etc/rc.d/init.d/nginx start;/etc/rc.d/init.d/redis start;/etc/rc.d/init.d/supervisord startStarting postgresql service... done.Starting nginx... done.Starting RedisRedis startedTraceback (most recent call last): File "/opt/freeware/bin/supervisord", line 33, in <module> sys.exit(load_entry_point('supervisor==4.2.0', 'console_scripts', 'supervisord')()) File "/opt/freeware/bin/supervisord", line 25, in importlib_load_entry_point return next(matches).load() File "/opt/freeware/lib64/python3.9/importlib/metadata.py", line 86, in load module = import_module(match.group('module')) File "/opt/freeware/lib64/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 680, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/opt/freeware/lib/python3.9/site-packages/supervisor/supervisord.py", line 41, in <module> from supervisor.options import ServerOptions File "/opt/freeware/lib/python3.9/site-packages/supervisor/options.py", line 13, in <module> import pkg_resourcesModuleNotFoundError: No module named 'pkg_resources'#
redis: 5.0.9-1aix
nginx: 1.21.4-2
postgresql: 11.19-1
supervisor: 4.2.0-3aix
I checked setuptools was installed (as Google stated it provides pkg_resources).
# python3 -m pip install setuptoolsRequirement already satisfied: setuptools in /usr/opt/python3/lib/python3.9/site-packages (58.1.0)
Here's my $PATH
# echo $PATH/usr/bin:/opt/freeware/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java8_64/jre/bin:/usr/java8_64/bin:/usr/local/bin:/sap/remote/aix/parkj3/scripts:/sap/remote/aix/_patch/scripts#
I upgraded setuptools
# python3 -m pip install --upgrade setuptoolsRequirement already satisfied: setuptools in /usr/opt/python3/lib/python3.9/site-packages (58.1.0)Collecting setuptools Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/4f/ab/0bcfebdfc3bfa8554b2b2c97a555569c4c1ebc74ea288741ea8326c51906/setuptools-68.1.2-py3-none-any.whl.metadata Downloading setuptools-68.1.2-py3-none-any.whl.metadata (6.2 kB)Downloading setuptools-68.1.2-py3-none-any.whl (805 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 805.1/805.1 kB 5.1 MB/s eta 0:00:00Installing collected packages: setuptools Attempting uninstall: setuptools Found existing installation: setuptools 58.1.0 Uninstalling setuptools-58.1.0: Successfully uninstalled setuptools-58.1.0Successfully installed setuptools-68.1.2WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv[notice] A new release of pip is available: 22.0.4 -> 23.2.1[notice] To update, run: python3 -m pip install --upgrade pip#
However I still get the same " ModuleNotFoundError: No module named 'pkg_resources' " error so I kept looking.
Running transaction Preparing : 1/1 Installing : python3.9-setuptools-65.5.1-1.noarch 1/2 Installing : python3-setuptools-65.5.1-1.noarch 2/2 Verifying : python3-setuptools-65.5.1-1.noarch 1/2 Verifying : python3.9-setuptools-65.5.1-1.noarch 2/2Installed: python3-setuptools-65.5.1-1.noarch python3.9-setuptools-65.5.1-1.noarchComplete!# /etc/rc.d/init.d/postgresql start;/etc/rc.d/init.d/nginx start;/etc/rc.d/init.d/redis start;/etc/rc.d/init.d/supervisord startStarting postgresql service... done.nginx daemon is already running with PID 16384498.Starting RedisRedis started#
Checking the status of supervisord I now see the below (have stopped/cleared logs/started, same result).
# /opt/freeware/bin/supervisorctl statusawx-config-watcher FATAL Exited too quickly (process log may have details)tower-processes:awx-callback-receiver FATAL Exited too quickly (process log may have details)tower-processes:awx-daphne FATAL Exited too quickly (process log may have details)tower-processes:awx-dispatcher FATAL Exited too quickly (process log may have details)tower-processes:awx-rsyslogd RUNNING pid 18350516, uptime 0:01:29tower-processes:awx-uwsgi FATAL Exited too quickly (process log may have details)tower-processes:awx-wsbroadcast FATAL Exited too quickly (process log may have details)#
Checking the logs:
awx-callback-receiver.log | awx-daphne.log | awx-dispatcher.log | awx-uwsgi.log | awx-wsbroadcast.logexec(): 0509-036 Cannot load program /var/lib/awx/venv/awx/bin/python3 because of the following errors: 0509-150 Dependent module libpython3.7m.so could not be loaded. 0509-022 Cannot load module libpython3.7m.so. 0509-026 System error: A file or directory in the path name does not exist.
Ok, installed python3.7 to overcome the above issue and also found I had to add in the "ulimit 65535" to the startup process for supervisor (as it was re-set to the default when the newer supervisor was installed). I also installed python3-jinja2-3.0.3-1.noarch in case AWX was needing that.
AWX now starts however I still have the below error so does AWX also need re-packaging?
ERROR: Ansible requires Jinja2 3.0 or newer on the controller. Current version: 2.10.1
Does this help?
Require Jinja2 3.0.0 by mkrizek · Pull Request #75881 · ansible/ansible
GitHub | remove preview |
 | Require Jinja2 3.0.0 by mkrizek · Pull Request #75881 · ansible/ansible | SUMMARY ci_complete ISSUE TYPE Feature Pull Request COMPONENT NAME many | View this on GitHub > |
|
|
Many thanks, Steve
------------------------------
Steve Munday
AIX, IBM i, HMC, PowerVM
Original Message:
Sent: Tue September 05, 2023 01:45 PM
From: Andrey Klyachkin
Subject: Ansible feature requests
Hi Steve,
could you please try if it works for you:
https://dl.power-devops.com/supervisor-4.2.0-3aix.aix7.3.noarch.rpm
------------------------------
Andrey Klyachkin
https://www.power-devops.com
Original Message:
Sent: Tue September 05, 2023 01:37 PM
From: Steve Munday
Subject: Ansible feature requests
Andrey,
Hi, seems I'm now "stuck" until the new supervisord package is available. Sorry to press, would it be possible to have this made available soon please?
Many thanks, Steve
------------------------------
Steve Munday
AIX, IBM i, HMC, PowerVM
Original Message:
Sent: Mon September 04, 2023 01:06 PM
From: Andrey Klyachkin
Subject: Ansible feature requests
Hi Steve,
unfortunately I can't check it.
Please don't remove any RPM packages! They don't play such a big role. AWX uses packages from its own virtual environment. I am still working on bringing the latest version of AWX to AIX, but it requires newer Postgresql (>12) and it is not there. I think I have just to recompile supervisord package for newer python. It should be easier...
------------------------------
Andrey Klyachkin
https://www.power-devops.com
Original Message:
Sent: Mon September 04, 2023 12:25 PM
From: Steve Munday
Subject: Ansible feature requests
Andrey,
Hi, with the latest Ansible / Ansible-core installed (on AIX 7.3.0.2) should AWX on AIX still work just fine?
When running a Project sync I get the following:
ERROR: Ansible requires Jinja2 3.0 or newer on the controller. Current version: 2.10.1
However it seems I already have 3.0.3 loaded.
# ansible --versionansible [core 2.14.2] config file = /etc/ansible/ansible.cfg configured module search path = ['/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /opt/freeware/lib/python3.9/site-packages/ansible ansible collection location = /var/lib/awx/venv/vendor/awx_ansible_collections executable location = /usr/bin/ansible python version = 3.9.17 (main, Jul 5 2023, 06:17:22) [GCC 10.3.0] (/opt/freeware/bin/python3.9) jinja version = 3.0.3 libyaml = True#
Here's the query results from rpm -qa
# rpm -qa | grep -i jinjapython3.9-jinja2-3.0.3-1.noarchpython3-jinja2-2.10.3-1.noarch#
More checking, I think we're back to needing the new(er) version of supervisor aren't we??
# dnf list python3-jinja*Last metadata expiration check: 6:33:44 ago on September 4, 2023 at 05:44:52 AM EDT.Installed Packagespython3-jinja2.noarch 2.10.3-1 @AIX_Toolbox_noarchAvailable Packagespython3-jinja2.noarch 3.0.3-1 AIX_Toolbox_noarch# dnf update python3-jinja2Last metadata expiration check: 6:34:11 ago on September 4, 2023 at 05:44:52 AM EDT.Error: Problem: package supervisor-4.2.0-2aix.noarch requires /opt/freeware/bin/python3_64, but none of the providers can be installed - cannot install both python3-3.9.16-1.ppc and python3-3.7.11-1.ppc - cannot install both python3-3.9.16-1.ppc and python3-3.7.12-1.ppc - cannot install both python3-3.9.16-1.ppc and python3-3.7.15-1.ppc - cannot install both python3-3.9.16-1.ppc and python3-3.7.15-2.ppc - package python3-markupsafe-2.1.2-1.ppc requires python3 >= 3.9.16, but none of the providers can be installed - cannot install both python3-3.9.16-2.ppc and python3-3.7.11-1.ppc - cannot install both python3-3.9.16-2.ppc and python3-3.7.12-1.ppc - cannot install both python3-3.9.16-2.ppc and python3-3.7.15-1.ppc - cannot install both python3-3.9.16-2.ppc and python3-3.7.15-2.ppc - cannot install both python3-3.9.17-1.ppc and python3-3.7.15-2.ppc - package python3-jinja2-3.0.3-1.noarch requires python3-markupsafe >= 2.1.2, but none of the providers can be installed - cannot install the best update candidate for package python3-jinja2-2.10.3-1.noarch - problem with installed package supervisor-4.2.0-2aix.noarch - cannot install both python3-3.9.17-1.ppc and python3-3.7.11-1.ppc - cannot install both python3-3.9.17-1.ppc and python3-3.7.12-1.ppc - cannot install both python3-3.9.17-1.ppc and python3-3.7.15-1.ppc(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)#
Or do I remove python3-jinja2-2.10.3-1.noarch which will also remove python3-markupsafe-1.1.1-1.ppc?
Many thanks, Steve
------------------------------
Steve Munday
AIX, IBM i, HMC, PowerVM
Original Message:
Sent: Tue August 29, 2023 07:29 AM
From: Andrey Klyachkin
Subject: Ansible feature requests
Hi guys,
thank you for providing Ansible 7.2.0 and Ansible Core 2.14.2 in the Toolbox. I'd like to ask you to add two AIX-specific features in the packages.
1. Ansible requires now UTF-8 locale. Can you please provide wrapper scripts for Ansible commands with the locales set?
The simplest script could be as:
#!/bin/shLANG=en_US.UTF-8 /opt/freeware/bin/ansible.orig "$@"
or even:
alias ansible="LANG=en_US.UTF-8 /opt/freeware/bin/ansible"
I understand it might be sometimes very difficult to guess the correct locales but imho it is better to fall down to en_US.UTF-8 if the administrator didn't setup a UTF-8 compatible locale.
2. Ansible RPM package is a set of Ansible collections. Can you please add IBM-own collections into it like ibm.power_aix, ibm.power_hmc, ibm.power_vios? Administrators can always update them if they need, but having them in the RPM package is a good starting point. They are very often used on IBM AIX with Ansible automation and it would be nice to have them from the very beginning with Ansible.
------------------------------
Andrey Klyachkin
https://www.power-devops.com
------------------------------