IcePy: sequence<long> issues migrating to Python 3

joshmoore

Under Python 3, the long basic type listed under the basic type mappings no longer exists. API methods which take a sequence<long> are now failing with:

self = ..., colNumbers = [0, 1, 2, 3, 4, 5], start = 0, stop = 1, _ctx = None

    def read(self, colNumbers, start, stop, _ctx=None):

E Ice.UnknownException: exception ::Ice::UnknownException E { E unknown = builtins.ValueError: invalid value for element 0 of sequence<string> E }

where the definition of read is:

            idempotent
            Data
                readCoordinates(omero::api::LongArray rowNumbers)
                throws omero::ServerError;

and of LongArray is simply:

        sequence<long> LongArray;

Turning those values into strings prints the expected exception of:

E       ValueError: invalid value for element 0 of sequence<long>

which for me suggests a case statement which is falling through to the string default.

I imagine Ice.Long would be one solution as with JavaScript. Additionally or alternatively, would it be possible for the conversion to be similarly flexible for longs as with floats? (cf. https://forums.zeroc.com/discussion/4450/icepy-accept-integers-where-floats-are-expected) Or even better, does a workaround exist? e.g. is there a way to use the TypeInfo definitions to generate a sequence that will be appropriately detected?

Thanks in advance,
~Josh

xdm

Hi Josh,

Sequence of longs should work with Python 3.x we test this in Ice/operations tests

https://github.com/zeroc-ice/ice/blob/3.7/python/test/Ice/operations/Twoways.py#L434

https://github.com/zeroc-ice/ice/blob/3.7/python/test/Ice/operations/Test.ice#L127

What is the Slice definition of read you show us the definition of readCoordinates but that is not the one being used.

You might be interested in https://doc.zeroc.com/ice/3.7/language-mappings/python-mapping/client-side-slice-to-python-mapping/python-mapping-for-sequences

Ice 3.7 have some improvements for marshal sequences of basic types using python buffer protocol

joshmoore

What is the Slice definition of read you show us the definition of readCoordinates but that is not the one being used.

Sorry, it's essentially the same:

            idempotent
            Data
                read(omero::api::LongArray colNumbers, long start, long stop)
                throws omero::ServerError;

Sequence of longs should work with Python 3.x we test this in Ice/operations tests

Thanks. I'll look through the examples and see if I can pointpoint a difference.

some improvements for marshal sequences

Am I right in assuming this would impact end users? If possible, I'd much appreciate a non-breaking solution.

~Josh

xdm

What is the Slice definition for Data?

joshmoore

class Column {

    string name;
    string description;

};

sequence<Column> ColumnArray;

class Data {

    long lastModification;
    omero::api::LongArray rowNumbers;
    ColumnArray columns;

};

xdm

I was looking at this definitions in https://github.com/ome/omero-blitz/blob/718d598a49c0a5db1e0a7752200d07f1ef9b472a/src/main/slice/omero/Tables.ice#L130

My understanding is that the error is not related to the rowNumbers param but with the marshaling for the Data::columns, can you dump the Data::colums to check if it has the expected values. I assuming here you are marshaling StringColumn and it contains and unexpected (not string) value.

joshmoore

@xdm: good morning & thanks. Apparently I had lost the forest for the tress. The problem was indeed server-side with bytes needing to be decoded to native strings.

All the best,
~Josh