Archived

This forum has been archived. Please start a new discussion on GitHub.

Logical error in unmarshalling Ice.Long in Ice for Javascript

There seems to be an error in the Ice.Long.toNumber logic - the bytes are loaded from the bytestream into Long.high and Long.low specifically as int32 (signed) through the javascript DataView.getInt32 method.

My problem is that I'm loading a timestamp, which is a reasonably big number but easy enough to represent in Javascript's 64-bit float. Frequently a timestamp will have the low int > 2**31, and this is being read as a negative number for "low" while "high" is positive. Then the second else block below is fired, and the value is returned as high * Long.HIGH_MASK (which is correct) + low (which is incorrectly negative).

I'm having trouble finding information about how the data is being serialized in the first place.

The method toNumber has a few places where it seems to be treating them as UInt32, though, as in
(line 1602 of Ice.js)
        toNumber: function()
        {
            if((this.high & Long.SIGN_MASK) != 0)
            {
                var low = ~this.low;
                var high = ~this.high;
                if(low < 0xFFFFFFFF)
                {
                    low += 1;
                }
                else
                {
                    low = 0;
                    high += 1;
                    if(high > Long.HIGH_MAX)
                    {
                        return Number.NEGATIVE_INFINITY;
                    }
                }
                return -1 * (high * Long.HIGH_MASK) + low;
            }
            else
            {
                if(this.high > Long.HIGH_MAX)
                {
                    return Number.POSITIVE_INFINITY;
                }
                return (this.high * Long.HIGH_MASK) + this.low;
            }
        }
    });

    //
    // (high & SIGN_MASK) != 0 denotes a negative number;
    // that is, the most significant bit is set.
    //
    Long.SIGN_MASK = 0x80000000;

    //
    // When converting to a JavaScript Number we left shift the
    // high word by 32 bits. As that isn't possible using JavaScript's
    // left shift operator, we multiply the value by 2^32 which will
    // produce the same result.
    //
    Long.HIGH_MASK = 0x100000000;

    //
    // The maximum value for the high word when coverting to
    // a JavaScript Number is 2^21 - 1, in which case all
    // 53 bits are used.
    //
    Long.HIGH_MAX = 0x1FFFFF;

This if(low < 0xFFFFFFFF) statement will always fire, even though the bits have been flipped in low, as it only can range from -2^31 to (+2^31 - 1) and can never be as high as 2^32 -1 (0xFFFFFFFF), so the "else" will never be reached.

Comments

  • xdm
    xdm La Coruña, Spain
    See https://doc.zeroc.com/display/Ice36/JavaScript+Mapping+for+Built-In+Types#JavaScriptMappingforBuilt-InTypes-long"]JavaScript Mapping for Long Integers it explains the encoding of 64 bit integer in JavaScript, basically we encode it as two 32bit words.

    Note we don't provide an API to convert a JavaScript Number to an Ice.Long.

    I do a quick test with Ice.Long and nodejs and toNumber seems to be working correctly only for positive numbers.
    > new Date().getTime().toString(16);
    '14be994b448'
    > new Ice.Long(0x14b, 0xe994b448).toNumber().toString(16)
    '14be994b448'
    
    > (new Date().getTime() * 1000).toString(16);
    '510887788bd40'
    > new Ice.Long(0x51088, 0x7788bd40).toNumber().toString(16)
    '510887788bd40'
    > 
    

    But the conversion of negative numbers is not, you can use this version.
    toNumber: function()
    {
        if((this.high & Long.SIGN_MASK) !== 0)
        {
            if(this.high === 0xFFFFFFFF && this.low !== 0)
            {
                return -(~this.low + 1);
            }
            var high = ~this.high + 1;
            if(high > Long.HIGH_MAX)
            {
                return Number.NEGATIVE_INFINITY;
            }
    
            return -1 * (high * Long.HIGH_MASK) + this.low;
        }
        else
        {
            if(this.high > Long.HIGH_MAX)
            {
                return Number.POSITIVE_INFINITY;
            }
            return (this.high * Long.HIGH_MASK) + this.low;
        }
    }
    
  • This still doesn't work -
    in your test cases you are instantiating an Ice.Long object directly in javascript using essentially unsigned int32 objects. For your first test case, the low 32-bit word 0xe994b448 would be read by Ice.js from the buffer (using getInt32) as the integer -376130488, not as the unsigned integer 3918836808 you might be expecting.

    Because of this sending your test long 0x14be994b448 would be received by javascript and converted to a number as (0x14b * 0x100000000) + (-376130488) which is incorrect.

    The interpretation of the low word as a signed int at read seems to be the root of the problem.
  • xdm
    xdm La Coruña, Spain
    You are right the JavaScript implementation of getLong is bogus it should use getUint32 instead
    getLong: function()
    {
        if(this._limit - this._position < 8)
        {
            throw new Error(__BufferUnderflowException__);
        }
        var v = new Long();
        v.low = this.v.getUint32(this._position, true);
        this._position += 4;
        v.high = this.v.getUint32(this._position, true);
        this._position += 4;
        return v;
    }
    
  • Alright! With both modifications (both to the "toNumber" method and to "getLong" method) it now works great and returns the correct number for both positive and negative (long) values over the wire.
  • xdm
    xdm La Coruña, Spain
    We will include these fixes in the upcoming 3.6.0 release