Hi Rob,

I am using your ZIP & Unzip services to compress and uncompress the files in my flows. If I use your zip am not able to uncompress those files. If I use winzip to compress and send those file to your uncompress its able to uncompress those files. Any idea why its happening. You mentioned in your unzip service that it can’t handle large files, how to handle large files. Do I need to use some other streams to handle large files?. Please help

Thanks for your help in this.

There is an error in the zipFiles code. It isn’t setting the ZipEntry sizes. I’ll post an update when I can.

The way to handle large files is to not pull the entire file, compressed or uncompressed, into memory–which is what the zipFiles and unZipFiles services do. There are couple of ways to do this.

1. Do everything in a single Java service. For zipping, modify zipFiles to accept a target zip filename. Instead of hooking the ZipOutputStream to a ByteArrayOutputStream, hook it to a FileOutputStream (which you might front with a BufferedOutputStream). Then you create a loop for each file to zip, read a block, write a block, etc.

For unzipping, change unZipFiles to write the files instead of creating an array of entries.

The original zipFiles was written for someone who want to send the resulting byte stream over a comm link, so that’s why it returns a byte array instead of writing it to disk.

2. For max flexibility, write services that mimic the java.util.zip classes and methods. The thing here is that you’d need to write service wrappers for java.io things too. This is probably overkill for most needs.

Hope this helps.

Rob,

Thanks for your info. Well I was the one who asked the output as Byte Array. The reason is we want to send this byte array to a servlet for some reason they wanted like this. Their argument is by doing this way we can save space on the server. If we write the zip file or write the files when we unzip it needs the directory cleanup and we need to schedule a service to do this. I am not sure this is the best way to save the space on server.

But I used the same zipFiles service to zip the files when I tried to open it in WinZip it worked fine but not on PKZIP. java.util.zip is supported on the PKZIP only later version of 2.6G I downloaded a latest version but it did not work, even with the unZip service i was not able to uncompress thats why I created one more thread on this.
Thanks for you help on this.

That’s right. I forgot it was you. Sorry 'bout that.

As I’m digging into the error, it may be the unzip service that has the problem. I’m checking into it…

A new Zip Sample package is available to address the unzip problem.

I’m not sure about why pkzip doesn’t work.

An Update to my last posting. We found an interesting thing with the PKZIP. But before going in to the pkzip stuffs, when we send out the Byte Array to Servlet the b2b is adding some tags to the byte array. Because the default output template was html. We converted the byte array to string with the same settings, now its not adding any tags to the compressed string. The additional tags were the problems cause.

The WINZIP takes the byte array as input (zip file), ignoring all other tags it looks for the byte array only. That’s why even the compressed files with some html tags were working fine with WINZIP. But in PKZIP it opens the file for uncompress and start reading from the begining of the file, because of the tags its not able to uncompress file.

Rob & all Thanks for your help.

Hi Rob,

I using your updated Unzip Service to Unzip the files. I found out one problem with that. When it tries to unzip the file the result file is created with only some 256 characters after that everything is filled with spaces. But the file size of both original file which is got zipped and the extracted file which has spaces are same. I dont know how to fix this problem. I am posting that code here.

String zipFilename = “”;
ZipFile zf = null;

// pipeline
IDataHashCursor pipelineCursor = pipeline.getHashCursor();
if ( pipelineCursor.first( “zipFilename” ) )
{
zipFilename = (String) pipelineCursor.getValue();
}
pipelineCursor.destroy();

Vector entryVector = new Vector();

try
{
zf = new ZipFile(zipFilename);

java.util.Enumeration entries = zf.entries(); 


ZipEntry ze; 
while(entries.hasMoreElements()) 
{ 
ze = (ZipEntry)entries.nextElement(); 
byte[] b = null; 

if(!ze.isDirectory()) 
{ 
long entrySize = ze.getSize(); 

int count;
if(entrySize > Integer.MAX_VALUE)
throw new ServiceException(ze.getName() + " expands to " + ze.getSize() + " and is too large for this service.");
b = new byte[(int)entrySize];
InputStream is = zf.getInputStream(ze);
is.read(b, 0, b.length);

    } 

IData entry = IDataFactory.create();
IDataHashCursor entryCursor = entry.getHashCursor();
entryCursor.last();
entryCursor.insertAfter(“fileName”, ze.getName());
entryCursor.last();
entryCursor.insertAfter(“isDirectory”, (new Boolean(ze.isDirectory())).toString());
entryCursor.last();
entryCursor.insertAfter(“outputBytes”, b);
entryCursor.destroy();
entryVector.add(entry);
}

}
catch (java.io.IOException ioe)
{
throw new ServiceException("IOException: " + ioe.getMessage());
}
finally
{
if(zf != null)
{
try
{
zf.close();

}
catch (IOException ioe)
{
// Ignore any exceptions from above close
}
zf = null;
}
}

// pipeline
IDataHashCursor pipelineCursor_1 = pipeline.getHashCursor();
pipelineCursor_1.last();
pipelineCursor_1.insertAfter( “entries”, entryVector.toArray());
pipelineCursor_1.destroy();

Thanks,
Muru.

Hi Rob,

I think I fixed that problem. I used BufferedInputStream to read the contents of byte array instead InputStream. Here is the new code its works fine now.

String zipFilename = “”;
ZipFile zf = null;

// pipeline
IDataHashCursor pipelineCursor = pipeline.getHashCursor();
if ( pipelineCursor.first( “zipFilename” ) )
{
zipFilename = (String) pipelineCursor.getValue();
}
pipelineCursor.destroy();

Vector entryVector = new Vector();

try
{
zf = new ZipFile(zipFilename);

java.util.Enumeration entries = zf.entries(); 


ZipEntry ze; 
while(entries.hasMoreElements()) 
{ 
ze = (ZipEntry)entries.nextElement(); 
byte[] b = null; 

BufferedInputStream is = null;

    if(!ze.isDirectory()) 
{ 
long entrySize = ze.getSize(); 

//if(entrySize > Integer.MAX_VALUE)
//throw new ServiceException(ze.getName() + " expands to " + ze.getSize() + " and is too large for this service.");
b = new byte[(int)entrySize];
//InputStream is = zf.getInputStream(ze);
is = new BufferedInputStream(zf.getInputStream(ze));
is.read(b, 0, b.length);

    } 

IData entry = IDataFactory.create();
IDataHashCursor entryCursor = entry.getHashCursor();
entryCursor.last();
entryCursor.insertAfter(“fileName”, ze.getName());
entryCursor.last();
entryCursor.insertAfter(“isDirectory”, (new Boolean(ze.isDirectory())).toString());
entryCursor.last();
entryCursor.insertAfter(“outputBytes”, b);
entryCursor.destroy();
entryVector.add(entry);
}

}
catch (java.io.IOException ioe)
{
throw new ServiceException("IOException: " + ioe.getMessage());
}
finally
{
if(zf != null)
{
try
{
zf.close();

}
catch (IOException ioe)
{
// Ignore any exceptions from above close
}
zf = null;
}
}

// pipeline
IDataHashCursor pipelineCursor_1 = pipeline.getHashCursor();
pipelineCursor_1.last();
pipelineCursor_1.insertAfter( “entries”, entryVector.toArray());
pipelineCursor_1.destroy();

Thanks for your help,
Muru.

Muru: This is quite strange. I have duplicated this error but I’m confused as to what the exact cause is. Using BufferedInputStream should have no impact on proper operation. BufferedInputStream is derived from InputStream and simply buffers the incoming data.

I’m checking into this and will post what I find. Of course if someone sees what boneheaded thing I did in the code, please let me know.

The “contract” for InputStream.read(buf, offset, length) isn’t what I thought it was (my C up-bringing has bitten me again). The contract does not provide for filling the specified buf with length bytes in all but EOF or error situations as I assumed. BufferedInputStream does.

Sorry for the bug. I’ll post a fix to the sample package soon.

Thanks for pointing this out and working through it.

Hi Rob,
I am not clear on this part of your explanation.

“The contract does not provide for filling the specified buf with length bytes in all but EOF or error situations”

Though I understand the above code is working with BufferedInputStream but not with InputStream. My understanding of the difference between the above two was in terms of buffering before actually performing the I/O operation.

Please explain.
Thanks.

I’ll rephrase (my initial wording was rather terrible):

The InputStream.read(buf, offset, length) contract says that implementers can return before filling buf with length bytes. Multiple calls to read may be required if you want to fill a buf.

BufferedInputStream.read(buf, offset, length) performs multiple reads on the underlying stream in order to fill buf with length bytes. It will read until buf is filled, EOF or an exception occurs.

You can look at the code for the various input stream objects. The default stream that ZipFile.getInputStream (which is an object of a inner class named ZipFileInputStream) returns does not fill bufs completely during read. It appears to read the first 512 bytes (256 chars) from the zip file, inflate those and return.

Let me know if this explanation isn’t sufficient.

I am all clear now !! Thanks a lot for making me understand it.

Thus, it depends on implementors of InputStream to fill up the buffer or not, which is exactly what is said in API documentation… “Subclasses are encouraged to provide a more efficient implementation of this method”

Thanks once again.

One more question still: “object of a inner class named ZipFileInputStream”

How did you arive at this information?? I was not able to deduce this piece of information from the API itself??

Thanks a lot.

• Saurabh

The source for the core Java packages can be downloaded from Sun.

Oh !! Thanks a lot for this Info.

• Saurabh.

Hi Rob,

I am using your Unzip service to uncompress the files in my flows.I created the java service.Now, how can i define the Input/Output (String type or…)?
How can i test the service?
I don´t know how i get the zip file, if i have to put it in some special folder, or send it to my Unzip service.
Thanks!

Hi Rob

does it work with .gz files?

thanx

Hi
I need to upzip a zip file, which will have an excel spreadsheet
the services written in webMethods are not working
could any one help me out

I appreciate your early help

Thanks
Ashok

Can you elaborate on “not working?”

How to request help to maximize useful responses

When i copy and paste the services (ex: PSUtilities:unzip )into another java service, it is not compiling properly, it is taking the imports of other java service in that folder, it’s now working properly
if i use copy file also, when i save it is showing irrelevant errors which are not this java service errors
will PSUtilities work for GZip too
Thanks,
Ashok

Glad you got the imports part figured out.

I do not know if the PSUtilities unzip service works for GZip.

Hi
I’m working on a service where I receive a doc file and have to rename it to a zip and then unzip to continue with the process.

My problem is

I could unzip a zip file successfully, but when I rename a doc to zip and try to unzip I get an error called “over subscribed dynamic bits length tree”

If you have any information to resolve this issue, please let me know

Thanks in advance
Ashok

A Google search for “[FONT=Arial]oversubscribed dynamic bits length tree” turns up some hits including [URL=“http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4188883”]http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4188883[/URL].[/font]

What JVM version are you using?

I’m using 142 version

How is the file being received? If via FTP, is binary transfer mode being used?

There was a problem with the rename code, when it renames a doc to zip file, the file in zip gets corrupted. so i changed the code of rename file from doc to zip and it’s working fine

so i came to know that, the error “over subscribed dynamic bits length tree”
also occurs when the file is corrupted

Thanks
Ashok

HI ROB,

can u plz tel me wat r the inputs and ouptputs of ur unzipping code that u provided.

Thanks,
Syed