Thanks for all the answer before.

I have this problem many times.

what I did is to read a big xml(about 50mb), and make some string replacement, then output it.so I need to use pub.string:bytesToString to change the input file to String.

It worked well before even with 111mb.

But today java.nio.BufferOverflowException comes out when it process any files bigger than 20mb. And the system usage goes to 90% at the same time. (I use the same input xml as before)

I didn’t make any change, but it just keeps giving me the exception. This happened before and when I changed encoding to UTF-8 it was solved. after that, it worked well also for the ISO-8859-1.

It is very nasty, in the last 1 month, I am keeping fighting with it. Some time it even works with 111mb. no matter what encoding it is. And when I test with small files(up to 20mb sometimes), everything works well. And when I think I get it done finally, after several days, it comes out again.

please help me if you know what caused that. what I don’t understand is why it works sometime but not all the time. and why it works last week but not today, I didn’t change anything. Thanks very much

I think it is some bugs with java getBytes.
I see some advices that the file should be always smaller than 16mb. But I can’t because it will have big files all the time. It’s impossible to reduce it everyday. so is there any other better solution?

I have seen this issue before. Though this is not a webM issue and more of a java issue… this issue happens if we use the pub.string:bytesToString…in our project we changed the approach of reading the entire file into memory and instead we used BufferedReader and read the file in parts and proceed. We had to write a custom java service to do this.

HTH,
Anitha

but sometime it can continuously work well with bigger files. why can’t it always work ?

Review the info in Java bug database. The error occurs in very specific situations, which explains why the problem appears to be intermittent for you.

It said the problem is with the getBytes function (inside BytestoString service) when the file size goes up to 16mb. Since the problem is only with the getBytes, I think the around way is to use some other function to transfer the bytes to String by some other way.

So I wrote the java service as below which do this job,

StringBuffer fileContent = new StringBuffer();
IDataCursor dataCursor = pipeline.getCursor();
try {
dataCursor.next(“Filename”);
String filename = (String)dataCursor.getValue();
FileInputStream fin = new FileInputStream(filename);
BufferedReader in = new BufferedReader(new InputStreamReader(fin,“ISO-8859-1”));
String l = “”;
while ((l = in.readLine()) != null) {
fileContent.append(l);
fileContent.append(System.getProperty(“line.separator”)); //size of line.separator=2
}
String string = fileContent.toString();
dataCursor.insertAfter(“string”,string);
in.close();
} catch (Exception e){}

It works with small files, but when the file size goes up to 50 mb. The whole things is in mass again.Even the Developer becomes unstable.

I have no idea how to do this now, The wm build-in service can’t do that correctly and java service also doesn’t help this out.

I am asking for help now.

As I know in the service getFile, there is a opition which output the file as stream. If the file is set to output as stream, it is possible to choose the buffer size. So what could this buffersize do?

And is there any service which we could directly change the stream into String

From the posts in the link above:

“The same thing happens with StringBuffer.append when the size of the string buffer hits the aforementioned limit”

Perhaps the easiest thing to do is upgrade your JVM. I’m not sure which build of 1.4.2 has the fix but a review of the source for 1.4.2_12-b03 shows that StringCoding uses double instead of float when calculating the new buffer size. The source for 1.4.2_06-b03 shows that float is still used, so the fix was introduced somewhere between _06 and _12.

For the code you posted, you’re probably using readLine when you don’t need to. If the intent of your process is to simply read the contents of a file, perform some replacements and then write out the result, then perhaps CharArrayReader in conjuction with StringBuffer may be more appropriate.

May I ask what sort of “string replacement” you’re going to do? Perhaps reading the entire file into memory isn’t necessary?

but what I use is 1.5.0, but it still have that problem

it is a xml file, the values for some specific fileds are not in correct format. so it needs to find out those fields and check whether the value is correct, if not, reform it.

I upgrade the JVM to 1.5.0_11-b03 both in server and developer side, but it doesn’t seem to help

By the way, does anyone know how to add JVM heap memory size in WM, I know the command is java -Xmx 512MB, but what is the classname?

I find the problem I think, the Developer has its own jvm inside the webmethod folder. so it always call that jvm, which is 1.4.2.
So the question is how to upgrade the JVM for webmethod

There is a .bat or .sh file that will start IS and Developer. Modify those. You don’t need to run java.exe yourself.

That question is answered in the installation docs.

Given your posts that indicate you’re unfamiliar with Developer and IS and where the JVM they use is located–are you sure you’re using 1.5? In IS Administrator, take a look at the About page to confirm the JVM that is being used.

thanks for your help. One more question, does the wm 6.1 support jre 1.5.0? or only 1.4.x series.

'cause I installed a 1.5.0 and specified the java_dir to C:\Program Files\Java\jre1.5.0_11 which is the 1.5.0 is installed. But the server doesn’t work at all.

Or we need to copy the jre into the jvm folder under webmethod6.1?

by the way, where could i find the install document you specified.

Thanks very much, I am newbie for wm 6.1, so I have plenty of questions.

IS 6.1 is not certified for 1.5.0 as far as I know. Only 1.4.2.

Modifying the batch file to point to the java_dir is workable. There is no need to copy the JVM/JRE under the IS directory.

Advantage has all the product documentation available for download. advantage.webmethods.com. You’ll need an account to login, but it is free as long as you’re working for a current wM customer.

“wm 6.1” is not a product.

IS 6.1 does not support the 1.5.x JRE. Support for this was added with IS 6.5 SP2, I believe but you’d need to check the installation documentation under supported versions for details.

The webMethods Installation Guide for the 6.1 products can be found on Advantage in the Bookshelf->Product Installation section.

Mark

IS 6.5 SP2 did indeed add support for 1.5.x JVM versions. See this post for details.

Mark