Ice 3.5.1: Patch to fix extremely poor IcePy stringify performance

grembogrembo Michael GmelinOrganization: Grem Equity GmbHProject: E-Commerce platformMember
IcePy's stringify functions uses IceUtilInternal::Output to convert Ice data types such as struct or exception to strings. I assume that it was never meant to process large amounts of data, since it's extremely slow.

I stumbled over this when a Python process caught an exception containing one 1MB sized string member. The exception was implicitly converted to a string by an exception handler deep inside of the stack. The symptom was a python process running at 100% CPU that couldn't handle any application requests. As such implicit conversions are not uncommon in python I consider this to be extremely suboptimal.

cat > <<EOF
module Test
  struct TestStruct
     string message;

cat > <<EOF
import Ice
import Test

str(Test.TestStruct("1" * 1000 * 1000)) # 1 million characters

time python

real    1m21.264s
user    1m19.846s
sys     0m0.032s

The patches replaces IceUtilInternal::Output in IcePy with a local helper class called PrintHelper . It received some basic testing, you might want to double check yourself just to be sure.

Performance after applying the patch:
time python

real    0m0.059s
user    0m0.024s
sys     0m0.033s

The patch assumes that you already applied If you didn't, apply that one first.

To apply the attached source patch to a patched Ice 3.5.1 source distribution:
cd Ice-3.5.1
patch -p1 </tmp/ice351_icepy_stringify.txt 
Sign In or Register to comment.