I don't know if it's another behaviour of the same issue, but today, I noticed that "while read" results aren't given out of the loop in bash 5.0.18.
Here my example:
>cat hga.mailsend.table
# This is a comment line
# Now an address line:
donald.duck@duckburg.demickey.mouse@nowhere.com>cat hga.sh
#!/bin/bash
echo "My bash version is ${BASH_VERSION}"
emma="will it be changed ?"
hugo="initial value before loop"
anna="initial value before loop"
cat hga.mailsend.table | while read myrec ; do
echo "myrec: $myrec"
#
emma=$myrec
echo "inside loop: emma=$emma"
#
hugo="$hugo + $myrec"
echo "inside loop: hugo=[$hugo]"
#
anna=$(echo "$hugo + $myrec")
echo "inside loop: anna=[$anna]"
done
#
echo "loopend: emma=[$emma]"
echo "loopend: hugo=[$hugo]"
echo "loopend: anna=[$anna]"
The result is:
>./hga.sh
My bash version is 5.0.18(1)-release
myrec: # This is a comment line
inside loop: emma=# This is a comment line
inside loop: hugo=[initial value before loop + # This is a comment line]
inside loop: anna=[initial value before loop + # This is a comment line + # This is a comment line]
myrec: # Now an address line:
inside loop: emma=# Now an address line:
inside loop: hugo=[initial value before loop + # This is a comment line + # Now an address line:]
inside loop: anna=[initial value before loop + # This is a comment line + # Now an address line: + # Now an address line:]
myrec:
donald.duck@duckburg.deinside loop: emma=
donald.duck@duckburg.deinside loop: hugo=[initial value before loop + # This is a comment line + # Now an address line: +
donald.duck@duckburg.de]
inside loop: anna=[initial value before loop + # This is a comment line + # Now an address line: +
donald.duck@duckburg.de +
donald.duck@duckburg.de]
myrec:
mickey.mouse@nowhere.cominside loop: emma=
mickey.mouse@nowhere.cominside loop: hugo=[initial value before loop + # This is a comment line + # Now an address line: +
donald.duck@duckburg.de +
mickey.mouse@nowhere.com]
inside loop: anna=[initial value before loop + # This is a comment line + # Now an address line: +
donald.duck@duckburg.de +
mickey.mouse@nowhere.com +
mickey.mouse@nowhere.com]
loopend: emma=[will it be changed ?]
loopend: hugo=[initial value before loop]
loopend: anna=[initial value before loop]
So, the read loop works correct, no hangup, but the values set in the loop are reset after "done".
If I change the while-pipe to "for myrec in $(cat hga.mailsend.table); do" , I get the variable contents outside of the loop, but as "for" split by words, it doesn't make sense.
It seems not related to file I/O, for example:
cat hga.endless.sh
#!/bin/bash
ctr=0
while hugo="init" ; do
echo "before: $hugo"
hugo="exit"
echo "after: $hugo"
let ctr=ctr+1
[[ $ctr -gt 10 ]] && exit
done
echo "The end is near..."
Result :
./hga.endless.sh
before: init
after: exit
before: init
after: exit
before: init
after: exit
before: init
after: exit
before: init
after: exit
before: init
after: exit
before: init
after: exit
before: init
after: exit
before: init
after: exit
before: init
after: exit
before: init
after: exit
...aborted by ctrl-c...
------------------------------
Achim Haag
------------------------------
Original Message:
Sent: Fri April 23, 2021 02:06 AM
From: SANGAMESH MALLAYYA
Subject: Issue with Bash read builtin in latest release
It seems bash-5.1 has changed the ways FIFOs are implemented, hence the patch which is causing this issue is no more required.
We will be to updating bash to 5.1 which fixes this issue and let you know once it's available on AIX Toolbox.
------------------------------
SANGAMESH
Original Message:
Sent: Wed April 21, 2021 09:22 AM
From: Mike Whitton
Subject: Issue with Bash read builtin in latest release
Thanks for taking the time to look into this.
I have also been communicating with one of the maintainers of bash and he feels that the way patch 17 was implemented in the IBM version is causing the problem, he feels that the FIFO's are not being reaped properly after being written to.
btw, I was able to recompile the mainline bash binaries with all of the patches upto 18 applied and it works on the same AIX server without issue. So I do believe this issue exists with the distribution of bash released by IBM...
Thanks
------------------------------
Mike Whitton
Original Message:
Sent: Wed April 21, 2021 03:29 AM
From: Achim Haag
Subject: Issue with Bash read builtin in latest release
Strange, running your script, it doesn't end, it stands on the last read:
<root@t0201:/home/00000217/testdir>./testhg2.sh
SGLIST=("testfile1" "testfile1" "testfile1" "testfile1")
+ SGLIST=("testfile1" "testfile1" "testfile1" "testfile1")
for sg in ${SGLIST[@]}
do
while read line
do
echo "$line"
done < <(ls -al ./ 2>&1)
done
+ for sg in ${SGLIST[@]}
+ read line
++ ls -al ./
+ echo 'total 32'
total 32
+ read line
+ echo 'drwxr-xr-x 2 root system 256 Apr 21 09:12 .'
drwxr-xr-x 2 root system 256 Apr 21 09:12 .
+ read line
+ echo 'drwxr-xr-x 7 00000217 staff 4096 Apr 21 09:04 ..'
drwxr-xr-x 7 00000217 staff 4096 Apr 21 09:04 ..
+ read line
+ echo '-rwxr-xr-x 1 root system 179 Apr 21 09:15 testhg2.sh'
-rwxr-xr-x 1 root system 179 Apr 21 09:15 testhg2.sh
+ read line
+ echo '-rwxr-xr-x 1 root system 378 Apr 21 09:26 testhga.sh'
-rwxr-xr-x 1 root system 378 Apr 21 09:26 testhga.sh
+ read line
+ echo '-rwxr-xr-x 1 root system 381 Apr 21 09:12 testhgb.sh'
-rwxr-xr-x 1 root system 381 Apr 21 09:12 testhgb.sh
+ read line
Now I modified it slightly and it works (run it 50 times):
<root@t0201:/home/00000217/testdir>cat testhga.sh
!/bin/bash
echo "bash version: ${BASH_VERSINFO}"
rpm -qa |grep bash
SGLIST=("testfile1" "testfile1" "testfile1" "testfile1")
i=0
for sg in ${SGLIST[@]}
do
let i=i+1
echo "Loop $i"
j=0
while read line
do
let j=j+1
echo "Loop $i MYLINE: $line"
done < <(ls -al ./ 2>&1)
done
<root@t0201:/home/00000217/testdir>kkk
<root@t0201:/home/00000217/testdir>./testhga.sh
./testhga.sh: line 1: !/bin/bash: No such file or directory
bash version: 5
bash-5.0.18-1.ppc
Loop 1
Loop 1 MYLINE: total 32
Loop 1 MYLINE: drwxr-xr-x 2 root system 256 Apr 21 09:12 .
Loop 1 MYLINE: drwxr-xr-x 7 00000217 staff 4096 Apr 21 09:04 ..
Loop 1 MYLINE: -rwxr-xr-x 1 root system 179 Apr 21 09:15 testhg2.sh
Loop 1 MYLINE: -rwxr-xr-x 1 root system 378 Apr 21 09:26 testhga.sh
Loop 1 MYLINE: -rwxr-xr-x 1 root system 381 Apr 21 09:12 testhgb.sh
Loop 2
Loop 2 MYLINE: total 32
Loop 2 MYLINE: drwxr-xr-x 2 root system 256 Apr 21 09:12 .
Loop 2 MYLINE: drwxr-xr-x 7 00000217 staff 4096 Apr 21 09:04 ..
Loop 2 MYLINE: -rwxr-xr-x 1 root system 179 Apr 21 09:15 testhg2.sh
Loop 2 MYLINE: -rwxr-xr-x 1 root system 378 Apr 21 09:26 testhga.sh
Loop 2 MYLINE: -rwxr-xr-x 1 root system 381 Apr 21 09:12 testhgb.sh
Loop 3
Loop 3 MYLINE: total 32
Loop 3 MYLINE: drwxr-xr-x 2 root system 256 Apr 21 09:12 .
Loop 3 MYLINE: drwxr-xr-x 7 00000217 staff 4096 Apr 21 09:04 ..
Loop 3 MYLINE: -rwxr-xr-x 1 root system 179 Apr 21 09:15 testhg2.sh
Loop 3 MYLINE: -rwxr-xr-x 1 root system 378 Apr 21 09:26 testhga.sh
Loop 3 MYLINE: -rwxr-xr-x 1 root system 381 Apr 21 09:12 testhgb.sh
Loop 4
Loop 4 MYLINE: total 32
Loop 4 MYLINE: drwxr-xr-x 2 root system 256 Apr 21 09:12 .
Loop 4 MYLINE: drwxr-xr-x 7 00000217 staff 4096 Apr 21 09:04 ..
Loop 4 MYLINE: -rwxr-xr-x 1 root system 179 Apr 21 09:15 testhg2.sh
Loop 4 MYLINE: -rwxr-xr-x 1 root system 378 Apr 21 09:26 testhga.sh
Loop 4 MYLINE: -rwxr-xr-x 1 root system 381 Apr 21 09:12 testhgb.sh
<root@t0201:/home/00000217/testdir>
------------------------------
Achim Haag
Original Message:
Sent: Tue April 20, 2021 12:57 PM
From: Mike Whitton
Subject: Issue with Bash read builtin in latest release
I downloaded the official GNU release, patched it to version 5.0.18, the same release IBM is distributing, and it is working fine. The GCC compilers and GNU Make were used to build this package.
There is possibly something wrong with the version that is being distributed on the IBM AIX Linux Toolbox website.
Thanks
------------------------------
Mike Whitton
Original Message:
Sent: Mon April 19, 2021 05:11 PM
From: Mike Whitton
Subject: Issue with Bash read builtin in latest release
In the latest release of Bash on the toolbox website (5.0 Patch 18) I am running into a problem with one of my scripts and have isolated the issue. I did downgrade to the previous version of Bash from the toolbox (5.0, no patches) and it works fine.
What is happening is when I run a command (such as 'ls -al /etc') with a while loop and read the line it works fine sometimes but sometimes it does not. When it does not work fine it hangs once it reaches the end of the input that is being fed into the loop, it should see the EOF and automatically terminate but it does not. This previously worked fine 100% of the time.
I trussed the output and this is what we are getting when we reach the end of the line:
kfcntl(2, F_GETFL, 0x0FFFFFFFFFFFFFE8) = 67110914
kioctl(0, 22528, 0x0000000000000000, 0x0000000000000000) Err#25 ENOTTY
lseek(0, 0, 1) Err#29 ESPIPE
kread(0, "\t", 1) (sleeping...)
When it is working the kread function does not sleep, it just moves onto the next iteration in the while loop.
Here is a sample piece of code to recreate the issue:
SGLIST=("testfile1" "testfile1" "testfile1" "testfile1")
for sg in ${SGLIST[@]}
do
while read line
do
echo $line
done < <(ls -al /etc 2>&1)
done
This particular sample code reliably fails every time. The piece of code I have written fails with the same basic setup one out of every four times or so.
Any thoughts on this?
Thanks
------------------------------
Mike Whitton
------------------------------