Originally posted by: floore

Hi everyone,

 

I'm having an issue when copying a file from a Windows mount point (mounted using cifs) to a local filesystem (a disk presented from a SAN). I have created a JFS2 filesystem on the local disk, and it is 400GB in size. The file I am copying is 200GB, but it keeps failing at 4GB with the following:

 

'Value too large to be stored in data type'

 

The limits on the user are set to be unlimited:

 

I changed these limits as a test to see if the copy would complete, because by now I am tearing my hair out as to why this is failing!

 

If anyone can offer some assistance it would be greatly appreciated...boss is breathing down my neck!

 

Regards,

 

Tom

Originally posted by: Wouter Liefting

Tom, I did some reading and there seemed to be some inherent limitation in the CIFS protocol fields that limited file sizes to 4GB (2^32). There may have been workarounds introduced for this, but I did find a couple of old APARs for AIX that seemingly have not been closed. So I can well imagine (but am not entirely sure) that the AIX implementation of CIFS would not be able to handle file sizes larger than 4 GB. What exact AIX version are you using?

However... Is this a one-time copy or are you going to do this regularly? If it's just a one-time thing, you might want to consider some alternative solutions:

- Run NFS on the Windows box and use NFS to copy it.

- Use a Linux box as a man-in-the-middle, with CIFS on one end and NFS on the other.

- Use FTP or SSH.

- Use smbclient (from the Samba suite) to transfer the file.

- Use a tool such as GSplit to split the file into chunks <4GB on Windows, transfer them and then cat them together on the AIX box. 

In any case, I'm pretty sure this would not be a JFS/JFS2 limitation, so I don't think messing about with ulimit or /etc/security/limits.conf is going to help you. But if you want to be sure the problem is not at the JFS2 end: dd if=/dev/zero of=largetestfile bs=1G count=200

 

Edited: It looks like the 4G limit is still present in AIX 7.1. See this page, note 1 (at the bottom):

http://pic.dhe.ibm.com/infocenter/aix/v7r1/index.jsp?topic=%2Fcom.ibm.aix.commadmn%2Fdoc%2Fcommadmndita%2Fsmbfs_mount.htm

Originally posted by: floore

Wouter,

 

Thanks for the response. I was hoping to use NFS, because this is an Oracle dump file. CIFS is not supported with Oracle Data Pump, but NFS is, so there would be no need for this copy to happen in the first place. Using FTP is probably the only way to go. SSH won't work from the Oracle box because it's a Windows box I'm copying from. Otherwise I'd be using rsync. There are also no other Linux/UNIX boxes available because they are moving from Windows so have none. I would consider building my own, then using that as the middle man, but we use a site-to-site VPN to connect to the client's machines, and we only have a 2MB line! Copying a 200GB file down that line and back again isn't really an option ;)

Samba, NFS and SSH will require installing software, which needs approval from change control first...that's the real issue. There is a standalone version of FileZilla I could use, so this is a viable option. I was just hoping it was some setting I'd missed out in my filesystem/lv/vg setup. If not I guess I could use FileZilla for a one-way SSH connection to the box and transfer it that way. I'm just attempting one more copy then I'll have to give up and give FTP a shot. I'll get back with the results.

 

Regards,

 

Tom

Originally posted by: Wouter Liefting

Tom, are you allowed to install software on AIX without going through a lengthy change process? In that case I might expand a bit on the Samba solution that I proposed.

Samba in principle is a server suite, so it installs on AIX, to let AIX function as a CIFS/SMB server. But together with the suite of server tools you also get a bunch of client tools. These are mainly for testing but work fine for certain client applications. One of them is smbclient, which provides an FTP-like client interface, but connects to a CIFS/SMB server instead of an FTP server.

So after you've installed Samba on AIX, you run smbclient on it, supply the NetBIOS name, share name and user credentials, and then you can do put and get to send and retrieve files from the Windows share. There's no need to configure a Samba server - all you need is the smbclient tool. 

The advantage of this is that smbclient is completely self-contained, with its own implementation of the SMB/CIFS protocol. It does not rely on the AIX CIFS kernel implementation like mount -v cifs does. So it may well include the SMB/CIFS workarounds that would allow you to transfer files >4GB. And it requires zero changes to the Windows box.

Originally posted by: floore

So I've begun copying the file from the server using FTP and it is working fine. We are at 5GB and progressing at a faster pace than copying over CIFS. It seems the answer is to use NFS and not CIFS, for both copying large files and for my Oracle Data Pump issue (the fundamental reason for writing this post).

Thanks for your help. Can you point me to the document stating the CIFS file size? It was originally my understanding that CIFS file size limits were only dependant upon the underlying filesystem (FAT32, NTFS etc.).

Regards,

Tom

Originally posted by: Wouter Liefting

I edited my original post. The link to the 4G file size is at the bottom.

Originally posted by: teletype

Is is really cifs related? Or is it filesystem related? Does the filesystem support big files?

Hope this helps

Originally posted by: floore

Thanks for the reply, but this is definitely not a filesystem issue. The filesystem being used is JFS2, the user has unlimited file size limits, and copying a large file to it over FTP worked fine. I even checksummed the file and it came back as an exact match. It seems it was a CIFS issue after all.

Bizarre that CIFS would have a limit when it comes to copying files from it, to a filesystem that can handle large files.