Hello everybody,

I have a small problem with sftp. We have couple of Integrations that do sftp to remote servers after processing the data. We are facing with the following error:

Error Message: I/O Exception running command: java.io.IOException: Cannot run program “sftp”: error=12, Not enough space

We have students requesting for Transcripts and when we process the request and use SFTP to drop the file on remote server. Once this error starts happening, all the Integrations that uses sftp are failing due to this.

SoftwareAG Resolution:
Problem:
I/O Exception running command: java.io.IOException: Cannot run program “sftp”: error=12, Not enough space

Resolution:
This particular error is an OS error. It usually involves swap space. The recommendation is to allocate at least swap equal to 4 x JVM_MAXIMUM_MEMORY or 2 x PHYSICAL RAM whichever is larger.

Increase the “swap space” should resolve the error.

NOTE: The above solution didn’t help us.

Part time Solution: We have to re-start the Integration Server whenever we have this kind of issue.

Did anybody has to face this kid of issue, if so please let us know how you were able to resolve the issue.

Thanks,
Priyatham Porika.

Please check on which side the swap space needs to be altered.

Is it on the IS-side or more likely on target (SFTP-Server) side?

HTH,
Holger

Also I would say please re-open the SAG ticket and troubleshoot more as it seems issue is back and not fully resolved may be involve your target OS team also in the call for the root cause analysis sake as it sounds to be purely an internal OS issue with the remote sftp server tself.

HTH,
RMG

Let me break down the problem a bit and perhaps it will help you find your issue.

Your code is calling a native program, namely sftp. Unlike the execution of code within the JVM itself (e.g. running a simple Flow or Java service), the execution of a native program requires resources (including thread(s) and memory) in addition to those that are already allocated to the Integration Server’s JVM. This is one reason why it’s better to stick with pure Java solutions over solutions that rely on making native calls.

This error indicates that you’re running out of physical space, meaning: the space already allocated to applications like the IS + the space needed to make these sftp calls > the physical amount of memory of the server.

This could mean a few things. For example:

• You need more physical memory (i.e. everything is configured correctly - your server is just not big enough)

• You are allocating too much memory to your IS (and other applications on the server - if any), leaving very little physical memory for these other native calls to execute

• You could have a memory leak which is gradually eating up your memory and eventually making you run out

• You could have hung threads which are holding on to the memory

With this information in mind, you now need to do some investigation to understand exactly what is happening.

• Start by looking at how much memory your box has and how much memory is allocated to your applications (e.g. IS)? If you sum up the maximum memory allocated for each application on the server, does it leave enough physical memory for these other native calls?

• Start monitoring your server’s memory. Does it get consumed more and more over time? If so, who is consuming the most memory? Is it normal or is there a leak?

• Start monitoring the processes/threads on your server. Do you see an accumulation of a certain type of process over time? Perhaps you start seeing many sftp processes executing simultaneously. If so, why? Are they hung? Perhaps a timeout was not set?

And by all means, as RMG recommended, go ahead and reopen the ticket with SAG.

Percio

Thanks for the adding more notes Percio.

To add to Percio’s point about “stick with pure java”, Integration Server does have built in SFTP services starting with 9.0. You should not have to call out to external services to get this capability.

We are in the process of upgrading to WM 9.7. I see the following services available in WM 9.7:

pub.client.sftp:login
pub.client.sftp:get
pub.client.sftp:put
pub.client.sftp:logout

Earlier release(WM 8.2) we were using pub.openssh:sftp, which I don’t like it. The above ones are really proper services to use. Session key, login, get/put do all the stuff and logout. I am planning on changing to the above services now as our upgrade progresses.

I didn’t give up yet, I still have the ticket open with SoftwareAG. I will keep you guys posted.

Thank you all for the valuable inputs.
Priyatham

Great news. Switching to a pure Java solution should definitely help. Did you try some of the suggestions made to help find the root cause?

Percio

The mentioned “pub.client.sftp:*” services have been introduced with 9.5.

We switched our implementation from the WmSSH2 package (a SAG Consulting Package) to the build-in services during our migration from 7.1 to 9.5.

Except for the public key authentication we got them working. We keep investigating the public key authentication as one of our partner systems requests this.

Both variants (the WmSSH2-Package and the build-in serivces) are using Jcraft Secure Channel as library.

Benefit of using the new build-in services is that they are covered by the support team, where the WmSSH2-Package and the openSSH-Package were not.

Regards,
Holger

We were able to resolve this issue by following the steps below (webMethods 8.2):

Here is the JVM parameter
-Djdk.lang.Process.launchMechanism=posix_spawn

And this is the Oracle patch page.

Now do not have any SFTP issues in WM 8.2. But like I mentioned before we are going with webMethods sftp services provided in WmPublic

Thanks all for your valuable time and inputs. Very much appreciate it!
Priyatham