IBM Z and LinuxONE IBM Z

  • 1.  Ansible zos core collection module zos_apf failure .

    Posted Thu September 07, 2023 09:29 AM

    While trying to list APF Datasets, I am getting this error .

    Sample Play and task failing at "name: APF AMI OPS db2 Datasets"
    tasks:
          #- name: Ping DB2A
          - name: Ping the z/OS host and perform resource checks
            zos_ping:
            register: result

          - name: Display PING result
            debug:
              var: result  

          - name: APF AMI OPS db2 Datasets
            zos_apf:
              operation: list
            register: myapf
    This is the error I am receiving.
    "
    TASK [APF AMI OPS db2 Datasets] **********************************************************************************************************************************************************************
    task path: /root/myenv/apf.yml:24
    Using module file /root/.ansible/collections/ansible_collections/ibm/ibm_zos_core/plugins/modules/zos_apf.py
    Pipelining is enabled.
    <db2a.xxx.com> ESTABLISH SSH CONNECTION FOR USER: mvsmkj
    <db2a.xxx.com> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o 'User="mvsmkj"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/0494b81ceb"' db2a.xxx.com '/bin/sh -c '"'"'ZOAU_ROOT=/shrd/zoautil ZOAU_HOME=/shrd/zoautil PATH=/shrd/zoautil/bin:/shrd/cyp/usr/lpp/IBM/cyp/v3r10/pyz/bin:/bin:. LIBPATH=/shrd/zoautil/lib:/shrd/cyp/usr/lpp/IBM/cyp/v3r10/pyz/lib:/lib:/usr/lib:. PYTHONPATH=/shrd/zoautil/lib _BPXK_AUTOCVT=ON /shrd/cyp/usr/lpp/IBM/cyp/v3r10/pyz/bin/python3 && sleep 0'"'"''
    <db2a.xxx.com> (1, b'', b"SyntaxError: Non-UTF-8 code starting with '\\xa2' in file <stdin> on line 2, but no encoding declared; see https://python.org/dev/peps/pep-0263/ for details\n")
    <db2a.xxx.com> Failed to connect to the host via ssh: SyntaxError: Non-UTF-8 code starting with '\xa2' in file <stdin> on line 2, but no encoding declared; see https://python.org/dev/peps/pep-0263/ for details
    fatal: [db2a.xxx.com]: FAILED! => {
        "changed": false,
        "module_stderr": "SyntaxError: Non-UTF-8 code starting with '\\xa2' in file <stdin> on line 2, but no encoding declared; see https://python.org/dev/peps/pep-0263/ for details\n",
        "module_stdout": "",
        "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
        "rc": 1
    }
    Any suggestion what is being done wrong here.
    Ansible Version
    ansible             8.3.0
    ansible-core        2.15.3
    Python Version Python 3.9.5
    Regards,
    ManoJ K Jadwani


    ------------------------------
    Manoj Jadwani
    Product Manager
    BMC Software
    ------------------------------


  • 2.  RE: Ansible zos core collection module zos_apf failure .

    Posted Fri September 08, 2023 02:18 AM

    Hi,

    It works for me if I set the environment like in the example below. However you need to have the PTF installed which provides support the PYTHONSTDINENCODING variable. See here for more details --> https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/demetrios-dimatos1/2023/04/21/ibm-ansible-zos-core-150-has-released

    ---
    - name: List APF
      hosts: all
      gather_facts: False

      vars:
        PYZ: /usr/lpp/python
        ZOAU: /usr/lpp/zoau

        environment_vars:
          _BPXK_AUTOCVT: "ON"
          ZOAU_HOME: "{{ ZOAU }}"
          PYTHONPATH: "{{ ZOAU }}/lib"
          LIBPATH: "{{ ZOAU }}/lib:{{ PYZ }}/lib:/lib:/usr/lib:."
          PATH: "{{ ZOAU }}/bin:{{ PYZ }}/bin:/bin:/var/bin"
          _CEE_RUNOPTS: "FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
          _TAG_REDIR_ERR: "txt"
          _TAG_REDIR_IN: "txt"
          _TAG_REDIR_OUT: "txt"
          LANG: "C"
          PYTHONSTDINENCODING: "cp1047"

      environment: "{{ environment_vars }}"

      tasks:
      - name: List APF
        ibm.ibm_zos_core.zos_apf:
          operation: list
        register: myapf

    ...



    ------------------------------
    Oliver Stadler
    ------------------------------



  • 3.  RE: Ansible zos core collection module zos_apf failure .

    Posted Tue September 12, 2023 11:26 AM

    Hi Oliver,

    Thanks for your response but still receiving the same error.

    This is my playbook

    zos_ping works .
    zos_apf fails with same error as reported earlier

    TASK [List APF] ******************************************************************************************************************************************************************************************************************
    task path: /root/myenv/newt.yml:25
    Using module file /root/.ansible/collections/ansible_collections/ibm/ibm_zos_core/plugins/modules/zos_apf.py
    Pipelining is enabled.
    <db2a.xxx.com> ESTABLISH SSH CONNECTION FOR USER: mvsmkj
    <db2a.xxx.com> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o 'User="mvsmkj"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/0494b81ceb"' db2a.xxx.com '/bin/sh -c '"'"'_BPXK_AUTOCVT=ON ZOAU_HOME=/shrd/zoau1.2.4 PYTHONPATH=/shrd/zoau1.2.4/lib LIBPATH=/shrd/zoau1.2.4/lib:/shrd/cyp/usr/lpp/IBM/cyp/v3r10/pyz/lib:/lib:/usr/lib:. PATH=/shrd/zoau1.2.4/bin:/shrd/cyp/usr/lpp/IBM/cyp/v3r10/pyz/bin:/bin:/var/bin PYTHONSTDINENCODING=cp1047 /shrd/cyp/usr/lpp/IBM/cyp/v3r10/pyz/bin/python3 && sleep 0'"'"''
    <db2a.xxx.com> (1, b'', b"SyntaxError: Non-UTF-8 code starting with '\\xa2' in file <stdin> on line 2, but no encoding declared; see https://python.org/dev/peps/pep-0263/ for details\n")
    <db2a.xxx.com> Failed to connect to the host via ssh: SyntaxError: Non-UTF-8 code starting with '\xa2' in file <stdin> on line 2, but no encoding declared; see https://python.org/dev/peps/pep-0263/ for details
    fatal: [db2a.xxx.com]: FAILED! => {
        "changed": false,
        "module_stderr": "SyntaxError: Non-UTF-8 code starting with '\\xa2' in file <stdin> on line 2, but no encoding declared; see https://python.org/dev/peps/pep-0263/ for details\n",
        "module_stdout": "",
        "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
        "rc": 1
    }

    PLAY RECAP ***********************************************************************************************************************************************************************************************************************
    db2a.xxx.com               : ok=1    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

    configuration changed since I first reported this

    zoaversion is 1.2.4  earlier it was 1.2.1

    python 3.10 (on z/os) 

    Ansible Version
    ansible             8.3.0
    ansible-core        2.15.3

    Collection                    Version
    ----------------------------- -------
    ibm.ibm_zos_core              1.6.0  
    From sysprog got to know Python on zos is not SMPE maintained , it is received pax.z form so could not list applied maintenance . 
    If I run same play as exercise on zos xplorer it works , there the zo au version 1.2 and python is 3.9 .
    Any suggestions will be helpful.
    Regards,
    Manoj K Jadwani



    ------------------------------
    Manoj Jadwani
    Product Manager
    BMC Software
    ------------------------------



  • 4.  RE: Ansible zos core collection module zos_apf failure .

    Posted Thu October 19, 2023 03:13 AM

    Hi Manoj, this appears this will require diagnosis, if you are able to, I suggest you engage the community who maintains the modules; you can start with a discussion or open an bug issue. The community has tooling that allows for bugs and discussions to be tracked, this forum is better for non diagnostic questions. Git issues and discussions are triaged weekly and then identified when they can be addressed, otherwise there is S&S available through Ansible Automation Platform. 

    Back to your question, if you review the community issues, you will see these 3 issues touching on the same problem which is your query to `list` is probably returning a character outside of the UTF-8 encoding meaning Python can't not understand what that character is position 2. 

    See:


    See also recent update to known issues:
    known_issues:
       - Several modules have reported UTF8 decoding errors when interacting with results
         that contain non-printable UTF8 characters in the response. This occurs when
         a module receives content that does not correspond to a UTF-8 value.
         These include modules `zos_job_submit`, `zos_job_output`,
         `zos_operator_action_query` but are not limited to this list.
         This will be addressed in `ibm_zos_core` version 1.10.0-beta.1.
         Each case is unique, some options to work around the error are below.
         - Specify that the ASA assembler option be enabled to instruct the assembler
           to use ANSI control characters instead of machine code control characters.
         - Add `ignore_errors:true` to the playbook task so the task error will not
           fail the playbook.
         - If the error is resulting from a batch job, add `ignore_errors:true` to the
           task and capture the output into a variable and extract the job ID with a
           regular expression and then use `zos_job_output` to display the DD without
           the non-printable character such as the DD `JESMSGLG`.
           (https://github.com/ansible-collections/ibm_zos_core/issues/677)
           (https://github.com/ansible-collections/ibm_zos_core/issues/776)
           (https://github.com/ansible-collections/ibm_zos_core/issues/972)
       - With later versions of `ansible-core` used with `ibm_zos_core` collection a
         warning has started to appear "Module "ansible.builtin.command" returned non UTF-8 data in the JSON response"
         that is currently being reviewed. There are no recommendations at this point.
         (https://github.com/ansible-collections/ibm_zos_core/issues/983)



    ------------------------------
    Demetri Dimatos
    IBM zOS Ansible Core Senior Technical Lead
    IBM
    San Jose CA
    ------------------------------



  • 5.  RE: Ansible zos core collection module zos_apf failure .

    Posted Fri October 20, 2023 02:51 AM

    ThankYou Demetri , I was able to figure this out . It was due to mismatch of release of Python and zosAU running on our system. 

    Once the compatibility was fixed was able to use the new PYTHONSTDINENCODING environment variable.

    On the issue forum , I suggested idea also for this , there should be some form of verification script to be available when Python on z/OS is upgraded a compatibility should be checked each time with other pieces /library etc.  so that overall  Ansible for z/OS framework should not be broken.



    ------------------------------
    Manoj Jadwani
    Product Manager
    BMC Software
    ------------------------------