Home› Comments

File datatype

nsnsnsns Member ✭✭
in Comments
Hi !

The transfer of files with data is rather typical task and maybe Ice can provide more help,
e.g. by extending the Slice language with new built-in type 'File' :

interface Transceiver {
File get();
void put(File bs);
};

The semantic of 'File' can be similar to 'sequence<byte>' (i.e. Unix model of files) but implementation can be done via memory mapping ('mmap'), i.e. without the 1MB limit.

Of course, this approach immediatly rises another question: binary vs text files.

Nevertheless, what do you think ?

--
Cheers, Nikolai

Comments

  • marcmarc FloridaAdministrators, ZeroC Staff Marc LaukienOrganization: ZeroC, Inc.Project: The Internet Communications Engine ZeroC Staff
    We rather don't put special-case types like this in Slice. A sequence of bytes works fine. It's not advisable to transfer such large amounts of data in one call anyway, as regardless of the mapping, Ice internally still has to copy the whole file into a buffer together with header and other parameters. (The complexity of a marshaling engine that tries to avoid this, sometimes also called "zero copy", is extreme, and not a good design trade-off -- but this is a different subject.)

    Have a look at how IcePatch is implemented, and how it transfers files in small chunks, all represented as a sequence of bytes.

    Note that the 1MB request size limit is not a hard limit. You can set it to any value you like by setting Ice.MessageSizeMax.
  • nsnsnsns Member ✭✭
    The transferring of big files by small chunks is exactly what I do in my application.
    This requires some coding because I need to maintain a session-id, and the support from
    Ice could be useful.
    Ice internally still has to copy the whole file into a buffer
    No, if File was a built-in type, Ice could take care to correctly and efficiently transfer the file
    by small chunks.

    --
    Cheers, Nikolai
  • marcmarc FloridaAdministrators, ZeroC Staff Marc LaukienOrganization: ZeroC, Inc.Project: The Internet Communications Engine ZeroC Staff
    Originally posted by nsns
    No, if File was a built-in type, Ice could take care to correctly and efficiently transfer the file
    by small chunks.

    I'm afraid it's not that easy. Without going into detail, this would require extensive changes to the Ice protocol and marshaling engine. The result would be a much more complex Ice core, only for the benefit of one special-purpose use-case, which can also be handled at the application level.
  • jinyi.smilodonjinyi.smilodon Member ✭
    I think it easy to use sequence of bytes to transfer file through programming extra code, and to support new data type may cause many problems.
Sign In or Register to comment.