Minimize copying of large sequence<byte> parameters

jberendsenjberendsen John BerendsenOrganization: Nvidia CorpProject: Distributed FilesystemMember

https://doc.zeroc.com/ice/3.7/language-mappings/c++11-mapping/client-side-slice-to-c++11-mapping/c++11-mapping-for-sequences

The above documentation link suggests using ["cpp:array"] Ice::ByteSeq instead of sequence to minimize copying of data, saying:

“the [std::pair<…>] pointers point directly into the server-side transport buffer; this allows the server-side run time to avoid creating a vector to pass to the operation implementation, thereby avoiding both allocating memory for the sequence and copying its contents into that memory.”

For our application, we’d like to extend the life-cycle of the passed-in data beyond the return of the servant method. I don’t quite understand why, for raw bytes, the server-side run time cannot dump the transport data directly into the dynamically allocated memory of the vector<> container, thus getting zero-copy performance while allowing efficient vector swap/move semantics to be used to extend the life-cycle of the passed-in data.

Going one step further with the std::vector<> class, it would be good if a specialized allocator_type could be specified for vector construction, which happens inside the ICE run time. It that possible?

Best Answers

  • bernardbernard Jupiter, FLBernard NormierOrganization: ZeroC, Inc.Project: Ice ZeroC Staff
    edited January 30 Accepted Answer

    Hi John,

    Ice reads the request's network data and puts it into an InputStream. It then uses this InputStream to unmarshal the parameters of your operation and dispatch the request to your servant; this unmarshaling and dispatch occurs typically in the generated code.

    There is currently no way for you to adopt the InputStream or its underlying buffer. Furthermore, when a request completes (usually because you provide a response or exception), this InputStream and its underlying buffer are deallocated, and there is no way for you to delay or cancel this deallocation.

    For your use-case, I suggest to use a std::vector or another container such a std::list. You can then keep/move this container, and there is just one "extra" memory allocation, when the memory is copied from the InputStream's buffer into your container.

    Regarding custom allocators, Ice C++ allows you to use a custom mapping with the cpp:type metadata. For example:

    ["cpp:type:std::vector<int, MyCustomAllocator>"] sequence<int> IntSeq;
    

    This allows you to use a custom allocator of your choice, or a completely custom container.

    See The cpp:type and cpp:view-type Metadata Directives for more details.

    Best regards,
    Bernard

  • bernardbernard Jupiter, FLBernard NormierOrganization: ZeroC, Inc.Project: Ice ZeroC Staff
    Accepted Answer

    Hi John,

    Yes, a future Ice for C++ could allow you to adopt the InputStream or (more likely) its byte buffer. It's a reasonable feature request and I suggest you create an enhancement issue on GitHub.

    On the other hand, bypassing the InputStream to unmarshal network data directly into your std::vector parameter sounds much less reasonable.

    With C++11 and up, we can now move memory in a safe and elegant manner. While the Ice C++11 mapping allows you to move/adopt the memory for parameters, I am sure there are other memory-moving opportunities we overlooked.

    Best regards,
    Bernard

Answers

  • bernardbernard Jupiter, FLBernard NormierOrganization: ZeroC, Inc.Project: IceAdministrators, ZeroC Staff ZeroC Staff
    edited January 30 Accepted Answer

    Hi John,

    Ice reads the request's network data and puts it into an InputStream. It then uses this InputStream to unmarshal the parameters of your operation and dispatch the request to your servant; this unmarshaling and dispatch occurs typically in the generated code.

    There is currently no way for you to adopt the InputStream or its underlying buffer. Furthermore, when a request completes (usually because you provide a response or exception), this InputStream and its underlying buffer are deallocated, and there is no way for you to delay or cancel this deallocation.

    For your use-case, I suggest to use a std::vector or another container such a std::list. You can then keep/move this container, and there is just one "extra" memory allocation, when the memory is copied from the InputStream's buffer into your container.

    Regarding custom allocators, Ice C++ allows you to use a custom mapping with the cpp:type metadata. For example:

    ["cpp:type:std::vector<int, MyCustomAllocator>"] sequence<int> IntSeq;
    

    This allows you to use a custom allocator of your choice, or a completely custom container.

    See The cpp:type and cpp:view-type Metadata Directives for more details.

    Best regards,
    Bernard

  • jberendsenjberendsen John BerendsenOrganization: Nvidia CorpProject: Distributed FilesystemMember

    Thank you Bernard,

    Your answer suggests that the zero-copy life-cycle extension through transfer of ownership is theoretically possible with some enhancement to the ZeroC library. Is that correct? If so, what argument would convince ZeroC to implement such an enhancement?

    Best regards,

    John.

  • bernardbernard Jupiter, FLBernard NormierOrganization: ZeroC, Inc.Project: IceAdministrators, ZeroC Staff ZeroC Staff
    Accepted Answer

    Hi John,

    Yes, a future Ice for C++ could allow you to adopt the InputStream or (more likely) its byte buffer. It's a reasonable feature request and I suggest you create an enhancement issue on GitHub.

    On the other hand, bypassing the InputStream to unmarshal network data directly into your std::vector parameter sounds much less reasonable.

    With C++11 and up, we can now move memory in a safe and elegant manner. While the Ice C++11 mapping allows you to move/adopt the memory for parameters, I am sure there are other memory-moving opportunities we overlooked.

    Best regards,
    Bernard

Sign In or Register to comment.