Icepatch2 client dies on checksum

Ric

One of our beta testers have a problem with my Icepatch2 based client when calculating checksum (ie. doing a thorough patch).
On my own computer there is no problem, and I haven't seen any other of our beta testers report the problem.

The problem manifests as follows:
1) He install the icepatch2 client software and a handfull of small files (no subdirs).
2) First run calculated checksum fine and it then downloads the rest of the installation files (500+ files, about 700 MB total, 5 levels deep directory structure).
3) If he now tries to calculate checksum by doing a thorough patch, the icepatch client gets an error when trying to calculate checksum.

His computer is running XP-Home, on a P4 with 1GB RAM + 1.5GB virtual RAM.

If he forces the thorough patch by removing the .sum checksum file, a new .sum file isn't even created, suggesting an error at a very early stage.

The icepatch client do get a checksumStart() callback, but then dies before getting a checksumEnd(). I can't currently see if it has gotten any checksumProgress() calls.

I'm running it in a try/catch, catching:

• catch(const Ice::Exception& ex)
• catch(const std::string& ex)
• catch(...)

An it ends up in "catch(...)", so I don't get much info about what went wrong.

Do you have a suggestion about what else I might catch() to get more info?

Have you an idea about what might cause this problem?

marc

I'm afraid without further debugging info, it's impossible to say what the problem might be. Can you run this in a debugger, so that you get the stack trace of the exception?

marc

As for the exception, you might also want to catch std::exception and const char*.

Ric

I've managed to reproduce the problem on my development computer and pinpoint it to Ice-3.0.1\src\IcePatch2\Util.cpp line 894:

Util.cpp 893: 
Util.cpp 894: 	    ByteSeq bytes(relPath.size() + buf.st_size);
Util.cpp 895: 	    copy(relPath.begin(), relPath.end(), bytes.begin());
Util.cpp 896:

The problem is a failed memory allocation that isn't caught, and simply passed to the calling process as:

System.Runtime.InteropServices.SEHException: 
  External component has thrown an exception.
   at IcePatch2.Patcher.prepare(Patcher* )
   at Launcher.Form1.patchThread()

I provoked the problem by calculating checksum of a 1.3GB file on a computer with 1.5GB physical RAM and 2GB virtual RAM, but several of our beta testers have reported the problem when calculating checksum of the 650MB file already in our distribution.

The failed-memory-allocation is not solved by increasing virtual memory (I tried with 4GB virtual memory).

Do you have any workarounds to this problem, or even better a code-patch to IcePatch2 that calculates checksum in smaller blocks?

I've reported another aspect of icepatch2 memory usage in this thread: http://www.zeroc.com/vbulletin/showthread.php?t=2052

marc

We can replace the single call to SHA1 in Util.cpp with SHA1_Init, SHA1_Update, and SHA1_Final. See:

http://www.openssl.org/docs/crypto/sha.html

However, the basic problem that IcePatch2 is not intended to be used for few, very large files, but rather for many small and medium sized files, remains. I therefore don't think that your use of IcePatch2 will be very efficient, even when the SHA1 problem is fixed. You might therefore consider writing a custom patch tool, that breaks up large binaries into smaller binaries before patching, and then puts the smaller binaries together again on the client side.

Ric

marc wrote:

We can replace the single call to SHA1 in Util.cpp with SHA1_Init, SHA1_Update, and SHA1_Final.

Yes, that would do the trick.
Will you (ZeroC) write the patch, or should I work on it myself?

marc wrote:

However, the basic problem that IcePatch2 is not intended to be used for few, very large files.

If all you got is a hammer, everything looks like a nail - (and Icepatch2 is such a nice hammer.)

But you're right, updating a huge file with Icepatch2 isn't very bandwidth efficient, so I've added a pre-patcher using xdelta diffs for updating the one huge file before using Icepatch2 for updating the distribution.

In that context Icepatch2 works perfectly for:

• Initial download of our distribution (500 small/medium files + 1 huge file).
• Downloading a diff-file for the pre-patcher's use.
• Verifying the result of pre-patching the huge file, and downloading a correct version of the huge file if pre-patching failed. (This has saved my a** several times already when I made a blunder ).
• Updating all the small/medium files.

I noticed a peculiarity when setting up a second icepatch2server for serving pre-patcher diff-files; I now have an icepatch2 server with 500 files, and an icepatch2server with 5 files, and fetching and checking the filelist from the server with only 5 files takes about 10 times as long as fetching and checking the filelist with 500 files.

The reason was that for every file in the local filelist (Icepatch2.sum) which wasn't also present in the remote filelist, icepatch2 sent a server.getFileInfoSeq to the server. With 150ms ping to the server 500 calls kind of adds up.

The workaround of adding all files from the 500-file distribution into the 5-file directory, worked in speeding up downloading and checking the filelist with a factor of 10. (A bit counter intuituitive though )

marc

Ric wrote:

Yes, that would do the trick.
Will you (ZeroC) write the patch, or should I work on it myself?

Actually, it's not quite as easy as I first thought, because we would have to do something similar for compression.

In any case, we will look into what it takes to change this, and get back to you.

Ric wrote:

If all you got is a hammer, everything looks like a nail - (and Icepatch2 is such a nice hammer.)

But you're right, updating a huge file with Icepatch2 isn't very bandwidth efficient, so I've added a pre-patcher using xdelta diffs for updating the one huge file before using Icepatch2 for updating the distribution.

In that context Icepatch2 works perfectly for:

• Initial download of our distribution (500 small/medium files + 1 huge file).
• Downloading a diff-file for the pre-patcher's use.
• Verifying the result of pre-patching the huge file, and downloading a correct version of the huge file if pre-patching failed. (This has saved my a** several times already when I made a blunder ).
• Updating all the small/medium files.

I noticed a peculiarity when setting up a second icepatch2server for serving pre-patcher diff-files; I now have an icepatch2 server with 500 files, and an icepatch2server with 5 files, and fetching and checking the filelist from the server with only 5 files takes about 10 times as long as fetching and checking the filelist with 500 files.

The reason was that for every file in the local filelist (Icepatch2.sum) which wasn't also present in the remote filelist, icepatch2 sent a server.getFileInfoSeq to the server. With 150ms ping to the server 500 calls kind of adds up.

The workaround of adding all files from the 500-file distribution into the 5-file directory, worked in speeding up downloading and checking the filelist with a factor of 10. (A bit counter intuituitive though )

I don't think this behavior is wrong or counter intuitive. The patch client checks all files that it has in its summary with the server (optimized, using partitions and checksums for these partitions). If you do not wish the client to check for these files, you must remove them form the summary file.

Ric

marc wrote:

Actually, it's not quite as easy as I first thought, because we would have to do something similar for compression.

You are in luck; BZ2_bzWriteOpen, BZ2_bzWrite and BZ2_bzWriteClose in IcePatch2::compressBytesToFile() works analogous to SHA1_Init, SHA1_Update, and SHA1_Final. http://www.bzip.org/1.0.3/html/hl-interface.html#bzwrite
Though it will probably require restructuring IcePatch2::compressBytesToFile() a bit.

marc wrote:

In any case, we will look into what it takes to change this, and get back to you.

Thanks a lot.

dwayne

Hi,

Yes, I am working on a patch now and will send it to you once it has passed internal testing.

Regards,
Dwayne

dwayne

Hi,

Attached is a patch for src/IcePatch2/Util.cpp for Ice 3.0.1. Please try it out and let us know if you have any further problems.

Regards,
Dwayne

Ric

Patch verified

Everything works great.

I have now incorporated your patch and distributed it to all our beta testers. So far I've received no bug reports - only praise for your work.

Thanks again, this fix made a big difference.

/Ric