Apologize for the delayed response Oliver, I am on PTO thus some days not able to access the internet.
I did try your example and had no issues with:
Original Message:
Sent: Wed December 07, 2022 04:16 PM
From: Oliver Stadler
Subject: Encoding and file editing with Ansible
The issue with the template module is that if the template has less than 8 characters it fails. 8 or more characters are fine. While it's unusual that a template would be so short I was still wondering if the underlying issue might cause problems in other situations as well.
With the following playbook you should be able to reproduce the issue:
---
- name: My first zOS playbook
hosts: all
gather_facts: True
tasks:
- name: Create template1
ansible.builtin.lineinfile:
path: /tmp/template1.j2
line: "Today is day {{ '{{' }} ansible_date_time.day {{ '}}' }}"
create: True
state: present
delegate_to: localhost
- name: Create template2
ansible.builtin.lineinfile:
path: /tmp/template2.j2
line: "day: {{ '{{' }} ansible_date_time.day {{ '}}' }}"
create: True
state: present
delegate_to: localhost
- name: Create file using template1
ansible.builtin.template:
dest: /tmp/file1
src: /tmp/template1.j2
- name: Create file using template2
ansible.builtin.template:
dest: /tmp/file2
src: /tmp/template2.j2
...
When I run it I get the following error for the short second template:
TASK [Create file using template2] ************************************************************************************************************************************************************
fatal: [sys01.example.com]: FAILED! => {"changed": false, "checksum": "9dcfaa96b7ca35278aaaf7a83a913e1c47dad8a4", "expected_checksum": "1d0489730f034093393de6a41a7b8ba792d66a86", "msg": "Copied file does not match the expected checksum. Transfer failed."}
------------------------------
Oliver Stadler
Original Message:
Sent: Wed December 07, 2022 12:32 PM
From: DEMETRIOS DIMATOS
Subject: Encoding and file editing with Ansible
Hello Oliver,
We do try to track issues that occur in the community modules as you noted and so that eventually we can we try to fix them. We have fixed a few issues with privilege escalation, unfortunately we are still a fairly new offering and trying to address what is missing for z/OS specifically and then eventually we will get to the community modules.
If you keep seeing issues in the community modules, don't hesitate to share that information in any medium, can be here or in our ibm_zos_core repository (eg discussions) , we need to track them so we can see what we can do about helping make that change.
To your second point, I agree ansible.builtin.template is very powerful, I was not aware of any issues, I will try to see if i can recreate or at least note the issue as it may impact us as well because we are investing in templating :) and sharing code with the engine (ansible-core). Our direction is slightly different; community module copy and template perform nearly identical function with the differentiation being one supports Jinja2 templates and one does not. Thus our position given we have several modules that could benefit from templating, we are investing this technology into zos_copy, zos_job_submit and maybe zos_fetch.
You can review and track that work with these 3 issues (3 because this is a fairly large item and likely will go beyond 3 sprints - 6 weeks).
This also relates to a request to support z/OS Symbols in module zos_job_submit where you can view that issue 532 and feel free to comment if you agree it will be satisfied by template support noted in 428, 487, 488.
Feel free to also engage issues in our community as well , we have a great team and they enjoy interacting with our users.
------------------------------
DEMETRIOS DIMATOS
IBM z/OS Ansible Core Senior Technical Lead
IBM
Original Message:
Sent: Wed December 07, 2022 07:26 AM
From: Oliver Stadler
Subject: Encoding and file editing with Ansible
Hi Demetrios,
Thanks a lot for your feedback and for open the issue in Git.
With "manually" I meant just changing the tagging using the chtag command. As you suggested we added this to the playbook. But it would be much nicer if it would be covered in the Ansible module.
One more module which we did some tests is ansible.builtin.template which is very powerful and useful. While it bascially works we had a case where it behaved somewhat strange (if the template has less then 8 characters it throws a checksum error). Is the template module something you will look at when start working on the community modules or are the plans to add a zos_template to the z/OS core collection?
------------------------------
Oliver Stadler
Original Message:
Sent: Wed December 07, 2022 04:59 AM
From: DEMETRIOS DIMATOS
Subject: Encoding and file editing with Ansible
Hi,
Regarding the first point using community module ansible.builtin.file , I don't see a way this can be done, looking at modules source they perform a open(b_path, 'wb').close() where:
"w" - Write - Opens a file for writing, creates the file if it does not exist
"b" - Binary - Binary mode
It is possible "t" - Text mode would have helped here, but there is no option to override this from what I can see.
To your second point, I am not clear on what is meant by manually in "encoding without having to manually change or remove the tagging" , if you mean manually by having to SSH and chtag or conv, then you could put this in your playbook instead and use the command, shell, raw modules to execute chtag or iconv.
The third point with the double entry when using ibm.ibm_zos_core.zos_lineinfile , this s a bug, I have opened a git issue , thank you for letting us know.
As for some alternatives, although not ideal, you could try using zos_encode or zos_copy.
When using zos_encode you will need an existing file to encode from, its not idea but you could use the file created by ansible.builtin.file.
If you want to use zos_copy you can use the content option but it will require minimally 1 space to work, thus you end up with a file that has one space in it.
---- hosts: zvm gather_facts: false environment: "{{ environment_vars }}" collections: - ibm.ibm_zos_core tasks: - name: Touch a file ansible.builtin.file: path: /tmp/ansible_file_from_builtin_file state: touch - name: Convert file encoding from controller file to USS zos_encode: src: /tmp/ansible_file_from_builtin_file dest: /tmp/ansible_file_from_zos_encode from_encoding: ISO8859-1 to_encoding: IBM-1047 - name: Copy inline content to a a USS file with one space zos_copy: content: ' ' dest: /tmp/ansible_file_from_zos_copy encoding: from: UTF-8 to: IBM-1047
t ISO8859-1 T=on -rw-r--r-- 1 BPXROOT OMVSGRP 0 Dec 7 01:26 ansible_file_from_builtin_filet IBM-1047 T=on -rw------- 1 BPXROOT OMVSGRP 1 Dec 7 01:26 ansible_file_from_zos_copyt IBM-1047 T=on -rw-r--r-- 1 BPXROOT OMVSGRP 0 Dec 7 01:26 ansible_file_from_zos_encode
Our current mission is to develop and service the IBM z/OS core collection , it will be some time before we are working on the community modules. Let us know if there is anything more we can do to help. Maybe one of my colleges might have another idea.
------------------------------
DEMETRIOS DIMATOS
IBM z/OS Ansible Core Senior Technical Lead
IBM
Original Message:
Sent: Mon December 05, 2022 09:55 AM
From: Oliver Stadler
Subject: Encoding and file editing with Ansible
Hi,
We recently did some testing with Ansible on Unix System Services to create and update files and hit some issues.
When we run this command:
$ ansible -i inventory test1 -m ansible.builtin.file -a "path=/tmp/ansible_file state=touch"
The resulting file on the target server is tagged with ISO8859-1 encoding:
$ ls -laT /tmp/ansible_file
t ISO8859-1 T=on -rw-r--r-- 1 USER1 SYS1 0 Dec 5 13:07 /tmp/ansible_file
While there are cases where this is ok, most of the files we deal with on the USS platofrm are IBM-1047 encoded. Is there a way to touch a file via Ansible with IBM-1047 encoding without having to manually change or remove the tagging?
We also did some test with the ibm.ibm_zos_core.zos_lineinfile module to update file content. Problem here is that when we run the command twice the line is also added twice to the file. Unlike the ansible.builtin.lineinfile module which doesn't add the line a second time if it is already there.
$ ansible -i inventory rplex -m ibm.ibm_zos_core.zos_lineinfile -a "path=/tmp/myfile line='ABCD'"
$ ansible -i inventory rplex -m ibm.ibm_zos_core.zos_lineinfile -a "path=/tmp/myfile line='ABCD'"
$ cat /tmp/myfile
ABCD
ABCD
------------------------------
Oliver Stadler
------------------------------