Originally posted by: Kostek

Hi All

I've got a script which sometimes runs ok and sometimes fails. I localized that the problem is with simple piece of code that uses subshell. Then I've created script that contains only those few lines and it fails(sometimes) as well.

This few lines remove trailing blanks from file in one step.

#!/bin/bash

cat file.txt | (rm file.txt ; touch file.txt ;
while read File_Line; do

echo ${File_Line} | sed 's/ *$//g' >> file.txt

done)

It sometimes fails with: cat: cannot open file.txt
The problem is that it not happens every time, few times it works fine and then is one time with error.

Do You know what is going on and what should be done to avoid this error?
AIX Version 6.1

cheers
kostek
#AIX-Forum

Originally posted by: SystemAdmin

I'm not exactly sure why that's happening. My first thought without attempting to duplicate the problem is that it might be a race condition. With that said, I'd accomplish what you're trying to do using Perl.

perl -wpi -e 's/\s+$//' file.txt
#AIX-Forum

Originally posted by: Kostek

Thanks for post, I'll consider to use perl if no way out.

Pleasy try to duplicate problem on You system, maybe then You will find what's up.

I duplicated the same error on linux debian, so i think it's not the AIX problem but rather something with subshell. I know this is strange but in this case it seems like sometimes command 'rm' in subshell is executed before 'cat' and then cat doesn't see file... ?
#AIX-Forum

Originally posted by: Holgervk

you are reading and writing to the same file
this gives unpredictable results. always avoid this.

you might do something like

cp file.txt file2.txt
cat file2.txt ...(rest of your code)
rm file2.txt
#AIX-Forum

Originally posted by: The_Doctor

As previously stated, this is a classic race condition between 2 different processes, giving unpredictable results.

The "cat" process is forked. The "rm" process is forked. These 2 forked processes run asynchronously.... not serially.... thus the unpredictable results.

Change your code to remove the reliance on "rm" & "touch" and you'll be good to go. aka.... there are many ways to do accomplish what you want without "rm" & "touch".

BTW.... your use of echo makes sed redundant. You can remove the pipe to sed & speed things up dramatically. Like I said.... many ways to do it, here's but one example:

#!/bin/ksh > temp.txt                         # ensure our temp file is empty 

while read File_Line 

do   echo $
{File_Line
} >> temp.txt   # sed 
's/ *$//g' ..... the string passed to sed is redundant # remove the pipe to sed and # avoid the fork of a 

new sed process 

for every line in the file # 1,000,000 lines = 1,000,000 forks = slow   done                  < file.txt   # or use the cat | format 

if you don
't like this   mv temp.txt file.txt               # destroy original file 

if you wish

#AIX-Forum

Originally posted by: SystemAdmin

For some reason I can't reply to The_Doctor's post despite trying three different browsers. He's of course right, but at the risk of sounding pedantic, it's not the echo that strips the line, but the read. A quick change of IFS and you'll see that the read no longer strips off the trailing white space, e.g.

$ cat print.bash
#!/bin/bash

IFS=':'
read LINE < $1
export LINE
perl -le 'print $ENV{LINE}' | od -c
$ od -c file.txt
0000000 t h i s i s a t e s t
0000020 \t \t \n
0000024
$ ./print.bash file.txt
0000000 t h i s i s a t e s t
0000020 \t \t \n
0000024
$ perl -wpi -e 's/IFS/#IFS/' print.bash
$ ./print.bash file.txt
0000000 t h i s i s a t e s t \n
0000017
#AIX-Forum

Originally posted by: The_Doctor

yeah, you're right

I believe it's:

• the read that removes the leading & trailing blanks from the complete line....
• while the echo (without double quotes) takes any consecutive imbedded blanks and reduces then down to 1 blank

For example:

echo 
"${File_Line}"           # retain any imbedded blanks with 

double quotes, 

while echo $
{File_Line
}            # the original echo takes 2 or more consecutive blanks & reduces them down to a single blank

#AIX-Forum

Originally posted by: Kostek

Thanks a lot guys

I didn't know that system treats those commands like two independent processes.
Thanks for performance tips as well.
#AIX-Forum