Originally posted by: SystemAdmin

Hi, we are writing large files (greater than 2GB) and find that in spite of having size_t size and numberOfItems parameters, the AIX libc function fwrite() complains of an invalid parameter if we use any combination that when multiplied together yield a value greater than 2GB. There is no mention of this limit in the man pages. On other platforms we have no problem writing 1 record of 3GB (on RHEL 4 for example). We compile with large file support enabled and fread() can in fact read the 3GB file in one call once it is written, but fwrite refuses to write more than (2GB-1) bytes per call.

Is this behavior as expected on AIX? We can code around the limitation, but it would be good to either have the documentation identify the limit of bytes written per call, or just change the c library routine to support writing full size_t amounts of data.

Regards,
Brion
#AIX-Forum

Originally posted by: hdkutz

Hello Brion,
maybe is there a ulimit setting for the user, which creates those files? (Check with ulimit -a at the users prompt)
Is the underlying Filesystem created as a big Filesystem? (Look at lsjfs or lsjfs2)

Cheers,
ku
#AIX-Forum

Originally posted by: SystemAdmin

ku,

thanks for your suggestion. The ulimits are all unlimited (except core file size). The filesystem is NFS. I don't think it has anything to do with the filesystem since we can write a 3GB file by looping and using fwrite to write 1GB or less per call. The fread will read a whole 3GB file in a single call, but if you attempt to write 2GB or more in one call to fwrite, it returns an error with errno showing it is a bad parameter. This must be because the fwrite library function is checking that the number of bytes to write must be less than 2GB. This in spite of the fact that the size parameter is size_t, which on a 64-bit AIX system is in fact 64 bits wide and should easily handle large files.

I can live with the limitation that fwrite can write at most 2GB-1 bytes in a single call, but such a limit should be acknowledged in documentation. Since other operating systems allow fwrite to write write more than 2GB in a single call, I think it would be best if the AIX libc fwrite function were updated to handle >2GB writes in a single call. This is just my opinion.

Regards,
Brion
#AIX-Forum