AIX Open Source

AIX Open Source

Share your experiences and connect with fellow developers to discover how to build and manage open source software for the AIX operating system

 View Only
Expand all | Collapse all

Ansible feature requests

  • 1.  Ansible feature requests

    Posted Tue August 29, 2023 07:29 AM

    Hi guys,

    thank you for providing Ansible 7.2.0 and Ansible Core 2.14.2 in the Toolbox. I'd like to ask you to add two AIX-specific features in the packages.

    1. Ansible requires now UTF-8 locale. Can you please provide wrapper scripts for Ansible commands with the locales set? 

    The simplest script could be as:

    #!/bin/sh
    
    LANG=en_US.UTF-8 /opt/freeware/bin/ansible.orig "$@"

    or even:

    alias ansible="LANG=en_US.UTF-8 /opt/freeware/bin/ansible"

    I understand it might be sometimes very difficult to guess the correct locales but imho it is better to fall down to en_US.UTF-8 if the administrator didn't setup a UTF-8 compatible locale.

    2. Ansible RPM package is a set of Ansible collections. Can you please add IBM-own collections into it like ibm.power_aix, ibm.power_hmc, ibm.power_vios? Administrators can always update them  if they need, but having them in the RPM package is a good starting point. They are very often used on IBM AIX with Ansible automation and it would be nice to have them from the very beginning with Ansible.



    ------------------------------
    Andrey Klyachkin

    https://www.power-devops.com
    ------------------------------


  • 2.  RE: Ansible feature requests

    Posted Tue August 29, 2023 08:48 AM

    Hi Andrey,

    Thanks for the suggestions. 

    We are already looking into the UTF-8 issue. We will upload a fixed version soon.

    For the second one, we will have to check some things before on our end and decide. 



    ------------------------------
    Ayappan P
    ------------------------------



  • 3.  RE: Ansible feature requests

    Posted Mon September 04, 2023 12:26 PM

    Andrey,

    Hi, with the latest Ansible / Ansible-core installed (on AIX 7.3.0.2) should AWX on AIX still work just fine?

    When running a Project sync I get the following:

    ERROR: Ansible requires Jinja2 3.0 or newer on the controller. Current version: 2.10.1

    However it seems I already have 3.0.3 loaded.

    # ansible --version
    ansible [core 2.14.2]
      config file = /etc/ansible/ansible.cfg
      configured module search path = ['/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
      ansible python module location = /opt/freeware/lib/python3.9/site-packages/ansible
      ansible collection location = /var/lib/awx/venv/vendor/awx_ansible_collections
      executable location = /usr/bin/ansible
      python version = 3.9.17 (main, Jul  5 2023, 06:17:22) [GCC 10.3.0] (/opt/freeware/bin/python3.9)
      jinja version = 3.0.3
      libyaml = True
    #

    Here's the query results from rpm -qa

    # rpm -qa | grep -i jinja
    python3.9-jinja2-3.0.3-1.noarch
    python3-jinja2-2.10.3-1.noarch
    #
    

    More checking, I think we're back to needing the new(er) version of supervisor aren't we??

    # dnf list python3-jinja*
    Last metadata expiration check: 6:33:44 ago on September  4, 2023 at 05:44:52 AM EDT.
    Installed Packages
    python3-jinja2.noarch                                                                      2.10.3-1                                                                       @AIX_Toolbox_noarch
    Available Packages
    python3-jinja2.noarch                                                                      3.0.3-1                                                                        AIX_Toolbox_noarch
    # dnf update python3-jinja2
    Last metadata expiration check: 6:34:11 ago on September  4, 2023 at 05:44:52 AM EDT.
    Error:
     Problem: package supervisor-4.2.0-2aix.noarch requires /opt/freeware/bin/python3_64, but none of the providers can be installed
      - cannot install both python3-3.9.16-1.ppc and python3-3.7.11-1.ppc
      - cannot install both python3-3.9.16-1.ppc and python3-3.7.12-1.ppc
      - cannot install both python3-3.9.16-1.ppc and python3-3.7.15-1.ppc
      - cannot install both python3-3.9.16-1.ppc and python3-3.7.15-2.ppc
      - package python3-markupsafe-2.1.2-1.ppc requires python3 >= 3.9.16, but none of the providers can be installed
      - cannot install both python3-3.9.16-2.ppc and python3-3.7.11-1.ppc
      - cannot install both python3-3.9.16-2.ppc and python3-3.7.12-1.ppc
      - cannot install both python3-3.9.16-2.ppc and python3-3.7.15-1.ppc
      - cannot install both python3-3.9.16-2.ppc and python3-3.7.15-2.ppc
      - cannot install both python3-3.9.17-1.ppc and python3-3.7.15-2.ppc
      - package python3-jinja2-3.0.3-1.noarch requires python3-markupsafe >= 2.1.2, but none of the providers can be installed
      - cannot install the best update candidate for package python3-jinja2-2.10.3-1.noarch
      - problem with installed package supervisor-4.2.0-2aix.noarch
      - cannot install both python3-3.9.17-1.ppc and python3-3.7.11-1.ppc
      - cannot install both python3-3.9.17-1.ppc and python3-3.7.12-1.ppc
      - cannot install both python3-3.9.17-1.ppc and python3-3.7.15-1.ppc
    (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
    #

    Or do I remove python3-jinja2-2.10.3-1.noarch which will also remove python3-markupsafe-1.1.1-1.ppc?

    Many thanks, Steve



    ------------------------------
    Steve Munday
    AIX, IBM i, HMC, PowerVM
    ------------------------------



  • 4.  RE: Ansible feature requests

    Posted Mon September 04, 2023 01:07 PM

    Hi Steve,

    unfortunately I can't check it.

    Please don't remove any RPM packages! They don't play such a big role. AWX uses packages from its own virtual environment. I am still working on bringing the latest version of AWX to AIX, but it requires newer Postgresql (>12) and it is not there. I think I have just to recompile supervisord package for newer python. It should be easier...



    ------------------------------
    Andrey Klyachkin

    https://www.power-devops.com
    ------------------------------



  • 5.  RE: Ansible feature requests

    Posted Mon September 04, 2023 01:18 PM

    Andrey,

    Agreed, I'll wait for the re-compiled supervisor then.  Any timeline for that?

    Many thanks, Steve



    ------------------------------
    Steve Munday
    AIX, IBM i, HMC, PowerVM
    ------------------------------



  • 6.  RE: Ansible feature requests

    Posted Tue September 05, 2023 01:38 PM

    Andrey,

    Hi, seems I'm now "stuck" until the new supervisord package is available.  Sorry to press, would it be possible to have this made available soon please?

    Many thanks, Steve



    ------------------------------
    Steve Munday
    AIX, IBM i, HMC, PowerVM
    ------------------------------



  • 7.  RE: Ansible feature requests

    Posted Tue September 05, 2023 01:46 PM
    Edited by Andrey Klyachkin Tue September 05, 2023 01:46 PM

    Hi Steve,

    could you please try if it works for you:

    https://dl.power-devops.com/supervisor-4.2.0-3aix.aix7.3.noarch.rpm



    ------------------------------
    Andrey Klyachkin

    https://www.power-devops.com
    ------------------------------



  • 8.  RE: Ansible feature requests

    Posted Wed September 06, 2023 04:31 AM
    Edited by Steve Munday Wed September 06, 2023 07:39 AM

    Andrey,

    Hi, good morning.

    Updating supervisor was successful.

    Downloading Packages:
    Running transaction check
    Transaction check succeeded.
    Running transaction test
    Transaction test succeeded.
    Running transaction
      Preparing        :                                                                                                                                                                     1/1
      Upgrading        : supervisor-4.2.0-3aix.noarch                                                                                                                                        1/2
      Running scriptlet: supervisor-4.2.0-3aix.noarch                                                                                                                                        1/2
    ln: failed to create symbolic link '/etc/rc.d/rc2.d/Ssupervisord': Do not specify an existing file.
    ln: failed to create symbolic link '/etc/rc.d/rc2.d/Ksupervisord': Do not specify an existing file.
    warning: %post(supervisor-4.2.0-3aix.noarch) scriptlet failed, exit status 1
    
    Error in POSTIN scriptlet in rpm package supervisor
      Running scriptlet: supervisor-4.2.0-2aix.noarch                                                                                                                                        2/2
      Cleanup          : supervisor-4.2.0-2aix.noarch                                                                                                                                        2/2
      Verifying        : supervisor-4.2.0-3aix.noarch                                                                                                                                        1/2
      Verifying        : supervisor-4.2.0-2aix.noarch                                                                                                                                        2/2
    
    Upgraded:
      supervisor-4.2.0-3aix.noarch
    
    Complete!
    #

    No more below errors :-)

    Problem: package supervisor-4.2.0-2aix.noarch requires /opt/freeware/bin/python3_64, but none of the providers can be installed

    I then ran dnf update which was successful.  Finally I stopped and started AWX and got the below during the restart.

    # /etc/rc.d/init.d/postgresql start;/etc/rc.d/init.d/nginx start;/etc/rc.d/init.d/redis start;/etc/rc.d/init.d/supervisord start
    Starting postgresql service... done.
    Starting nginx... done.
    Starting Redis
    Redis started
    Traceback (most recent call last):
      File "/opt/freeware/bin/supervisord", line 33, in <module>
        sys.exit(load_entry_point('supervisor==4.2.0', 'console_scripts', 'supervisord')())
      File "/opt/freeware/bin/supervisord", line 25, in importlib_load_entry_point
        return next(matches).load()
      File "/opt/freeware/lib64/python3.9/importlib/metadata.py", line 86, in load
        module = import_module(match.group('module'))
      File "/opt/freeware/lib64/python3.9/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
      File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 850, in exec_module
      File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
      File "/opt/freeware/lib/python3.9/site-packages/supervisor/supervisord.py", line 41, in <module>
        from supervisor.options import ServerOptions
      File "/opt/freeware/lib/python3.9/site-packages/supervisor/options.py", line 13, in <module>
        import pkg_resources
    ModuleNotFoundError: No module named 'pkg_resources'
    #

    redis: 5.0.9-1aix

    nginx: 1.21.4-2

    postgresql: 11.19-1

    supervisor: 4.2.0-3aix

    I checked setuptools was installed (as Google stated it provides pkg_resources).

    # python3 -m pip install setuptools
    Requirement already satisfied: setuptools in /usr/opt/python3/lib/python3.9/site-packages (58.1.0)

    Here's my $PATH

    # echo $PATH
    /usr/bin:/opt/freeware/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java8_64/jre/bin:/usr/java8_64/bin:/usr/local/bin:/sap/remote/aix/parkj3/scripts:/sap/remote/aix/_patch/scripts
    #

    I upgraded setuptools

    # python3 -m pip install --upgrade setuptools
    Requirement already satisfied: setuptools in /usr/opt/python3/lib/python3.9/site-packages (58.1.0)
    Collecting setuptools
      Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/4f/ab/0bcfebdfc3bfa8554b2b2c97a555569c4c1ebc74ea288741ea8326c51906/setuptools-68.1.2-py3-none-any.whl.metadata
      Downloading setuptools-68.1.2-py3-none-any.whl.metadata (6.2 kB)
    Downloading setuptools-68.1.2-py3-none-any.whl (805 kB)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 805.1/805.1 kB 5.1 MB/s eta 0:00:00
    Installing collected packages: setuptools
      Attempting uninstall: setuptools
        Found existing installation: setuptools 58.1.0
        Uninstalling setuptools-58.1.0:
          Successfully uninstalled setuptools-58.1.0
    Successfully installed setuptools-68.1.2
    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
    
    [notice] A new release of pip is available: 22.0.4 -> 23.2.1
    [notice] To update, run: python3 -m pip install --upgrade pip
    #

    However I still get the same " ModuleNotFoundError: No module named 'pkg_resources' " error so I kept looking.

    Running transaction
      Preparing        :                                                                                                                                                                     1/1
      Installing       : python3.9-setuptools-65.5.1-1.noarch                                                                                                                                1/2
      Installing       : python3-setuptools-65.5.1-1.noarch                                                                                                                                  2/2
      Verifying        : python3-setuptools-65.5.1-1.noarch                                                                                                                                  1/2
      Verifying        : python3.9-setuptools-65.5.1-1.noarch                                                                                                                                2/2
    
    Installed:
      python3-setuptools-65.5.1-1.noarch                                                           python3.9-setuptools-65.5.1-1.noarch
    
    Complete!
    # /etc/rc.d/init.d/postgresql start;/etc/rc.d/init.d/nginx start;/etc/rc.d/init.d/redis start;/etc/rc.d/init.d/supervisord start
    Starting postgresql service... done.
    nginx daemon is already running with PID 16384498.
    Starting Redis
    Redis started
    #

    Checking the status of supervisord I now see the below (have stopped/cleared logs/started, same result).

    # /opt/freeware/bin/supervisorctl status
    awx-config-watcher                      FATAL     Exited too quickly (process log may have details)
    tower-processes:awx-callback-receiver   FATAL     Exited too quickly (process log may have details)
    tower-processes:awx-daphne              FATAL     Exited too quickly (process log may have details)
    tower-processes:awx-dispatcher          FATAL     Exited too quickly (process log may have details)
    tower-processes:awx-rsyslogd            RUNNING   pid 18350516, uptime 0:01:29
    tower-processes:awx-uwsgi               FATAL     Exited too quickly (process log may have details)
    tower-processes:awx-wsbroadcast         FATAL     Exited too quickly (process log may have details)
    #

    Checking the logs:

    awx-callback-receiver.log | awx-daphne.log | awx-dispatcher.log | awx-uwsgi.log | awx-wsbroadcast.log
    exec(): 0509-036 Cannot load program /var/lib/awx/venv/awx/bin/python3 because of the following errors:
            0509-150   Dependent module libpython3.7m.so could not be loaded.
            0509-022 Cannot load module libpython3.7m.so.
            0509-026 System error: A file or directory in the path name does not exist.

    Ok, installed python3.7 to overcome the above issue and also found I had to add in the "ulimit 65535" to the startup process for supervisor (as it was re-set to the default when the newer supervisor was installed).  I also installed python3-jinja2-3.0.3-1.noarch in case AWX was needing that.

    AWX now starts however I still have the below error so does AWX also need re-packaging?

    ERROR: Ansible requires Jinja2 3.0 or newer on the controller. Current version: 2.10.1

    Does this help?

    Require Jinja2 3.0.0 by mkrizek · Pull Request #75881 · ansible/ansible

    GitHub remove preview
    Require Jinja2 3.0.0 by mkrizek · Pull Request #75881 · ansible/ansible
    SUMMARY ci_complete ISSUE TYPE Feature Pull Request COMPONENT NAME many
    View this on GitHub >

    Many thanks, Steve
    ------------------------------
    Steve Munday
    AIX, IBM i, HMC, PowerVM
    ------------------------------



  • 9.  RE: Ansible feature requests

    Posted Fri September 08, 2023 11:41 AM

    Andrey,

    Hi, any thoughts on this please?  I can't see where AWX is finding Jinja2 2.10 when rpm -qa has 3.0.3 clearly listed.

    Many thanks, Steve



    ------------------------------
    Steve Munday
    AIX, IBM i, HMC, PowerVM
    ------------------------------



  • 10.  RE: Ansible feature requests

    Posted Sat September 09, 2023 03:58 PM

    Hi Steve,

    yes, it looks like AWX must be repackaged with new Python 3.9 too. I am not sure that I will be able to do it next week.



    ------------------------------
    Andrey Klyachkin

    https://www.power-devops.com
    ------------------------------



  • 11.  RE: Ansible feature requests

    Posted Mon September 25, 2023 09:58 AM

    Andrey,

    Hi, good afternoon.

    I was wondering if you had a timeline for the re-packaging of AWX so it uses Python3.9 so that this Jinja2 3.0.3 issue can be resolved?

    Many thanks, Steve



    ------------------------------
    Steve Munday
    AIX, IBM i, HMC, PowerVM
    ------------------------------



  • 12.  RE: Ansible feature requests

    Posted Thu September 28, 2023 07:41 AM

    Andrey,

    Hi, just a thought, can we overcome the jinja2-3.0.3 issue by doing the following?

    Copy the contents of

    /opt/freeware/lib/python3.9/site-packages/jinja2

    Over the top of

    /var/lib/awx/venv/ansible/lib/python3.7/site-packages/jinja2

    Or am I over simplifying things too much?

    Many thanks, Steve



    ------------------------------
    Steve Munday
    AIX, IBM i, HMC, PowerVM
    ------------------------------



  • 13.  RE: Ansible feature requests

    Posted Thu September 28, 2023 07:50 AM

    Steve,

    I can't test it. But I don't think it'll work. 

    I am still by rebuilding of AWX. It requires a lot of time and lot of patches. The newest versions of AWX require some packages which are not available for AIX. The older version of AWX, which you have, don't work very well with Python 3.9 and some other newer versions of Python packages. I can't give you any timeline, sorry. 



    ------------------------------
    Andrey Klyachkin

    https://www.power-devops.com
    ------------------------------



  • 14.  RE: Ansible feature requests

    Posted Thu September 28, 2023 08:15 AM

    Andrey,

    Hi, understood, a complex issue to resolve.

    I'm wondering if a forced downgrade of ansible (so removing ansible-core etc.) would return things to a more stable situation.  I think that in doing this, though, would see loads of packages removed etc. etc.

    Many thanks, Steve



    ------------------------------
    Steve Munday
    AIX, IBM i, HMC, PowerVM
    ------------------------------



  • 15.  RE: Ansible feature requests

    Posted Tue April 09, 2024 01:21 PM

    Update

    Circling back to this issue.

    I tracked down where the jinja2 (AWX) package was located -- /var/lib/awx/venv/awx/lib/python3.7/site-packages -- and went ahead with resetting the (test) AWX environment by re-building from scratch allowing python3.9, ansible-core, and the like to get installed.

    I then renamed the existing (AWX) jinja2 directory(s) before copying in a jinja 3.1.3 directory structure, stopped/started AWX, and successfully connected.  So far so good!  However, when running a Job Template I still hit the issue whereby AWX "thinks" it's still got jinja2 2.10.1 installed so conclude there must be some "master list" of (AWX) packages which are referred to.

    If anyone know where this list might be or, in fact, whether there's (even) more needing to copied in, jinja2 wise, I'd be very interested to hear.

    Thanks, Steve



    ------------------------------
    Steve Munday
    AIX, IBM i, HMC, PowerVM
    ------------------------------



  • 16.  RE: Ansible feature requests

    Posted Tue September 05, 2023 10:30 AM

    On request #2 above, AIX follows the community model for Ansible content by making it freely available without support on Galaxy and available via Red Hat Automation Hub as certified and supported content. There is no intent to package and distribute the AIX Ansible collection outside of Galaxy and Automation Hub.



    ------------------------------
    Carl Burnett
    DE, IBM Systems Group, IBM Power
    ------------------------------



  • 17.  RE: Ansible feature requests

    Posted Tue September 05, 2023 01:03 PM

    Just to add to that thought, I've adopted Jeff Geerling's suggestion to import the collections you need directly into your git tree. He makes a compelling case and it's been working well for us.

    https://www.jeffgeerling.com/blog/2020/ansible-best-practices-using-project-local-collections-and-roles



    ------------------------------
    Mario
    ------------------------------