Originally posted by: Kostek

Hi All

Is there any way to combine(add) two text files without reading every line of each file ?
I need to add header at the top of file with data(few or more millions of records) so I would like to avoid reading each line with e.g.`cat`

Thanks in advance
kostek
#AIX-Forum

Originally posted by: ColombianJoker

Something like cat file1 file2 > file3 ? Not, that's the way, but is as fast as your system is.
#AIX-Forum

Originally posted by: Kostek

ColombianJoker thanks for reply

I've tried `cat` before and it took 14 minutes(10 million records in file - 3,7GB) - it's too long.

cheers
kostek
#AIX-Forum

Originally posted by: Kostek

orphy thanks for reply

The perl code You provided is exactly what I need but it takes the same time like `cat` - it means that it also reads every each line I guess.
In fact I need only to add header to one large file, I don't have two large files. That was only my idea to have header in another file
and then combine two files(header and data) without reading them.

So orphy Your idea looks good but it would be perfect 'tell' perl to stop processing file after adding first line.
kostek
#AIX-Forum

Originally posted by: ColombianJoker

If your header is exactly the same size than your disk device segments, there is a programmatically way to speed up your copy -yes it is a copy-, but, if not... sorry, you can't.
Speed up your system, and you will speed up your copies.
Optimize your I/O system, check your ioo and vmo settings.
#AIX-Forum

Originally posted by: orphy

Any reason why you don't like cat? some people think they are cute!

Regardless of what you use, the contents of the files will have to be read to be merged somehow although, depending on how you do it, one of them might just need of seek.

Here's a quick way to do it or you can just can it all the way...

1. cat o1

file 1, line 1
file 1, line 2
file 1, line 3

1. cat o2

file 2, line 1
file 2, line 2
file 2, line 3

1. perl -pi -e 'print "add me to top of file\n" if $. == 1' o1
2. cat o1

add me to top of file
file 1, line 1
file 1, line 2
file 1, line 3

1. cat o1 o2 >> o3
2. cat o3

add me to top of file
file 1, line 1
file 1, line 2
file 1, line 3
file 2, line 1
file 2, line 2
file 2, line 3

Or if you don't care about preserving the original files, simply run the perl o1 and then cat o2 >> o1. That will save you an inode!
Orphy
#AIX-Forum