Archived

This forum has been archived. Please start a new discussion on GitHub.

Get a node readable stream from server (backpressure)

sulliwane
edited July 2016 in Help Center

Hi,

Currently my node.js ice client do a simple data query to ice server, and the server uses a callback to send me back a LOT of javascript objects (so one by one through the callback function).

The problem with this method is I have no natural way to slow down the data flow (in case my node.js client process very slowly the arriving data, for example).

Knowing that node.js streams implement this backpressure mechanism, is there a way to receive my data from the ice server by reading a "readable stream", instead of having the ice server pushing the data to me, through callback.

Many thanks for your help! :)

Note: I looked at Ice streams for javascript (starting on 3.7), but the stream implementation doesn't follow node.js streams (so no .pipe() facility I suppose). I'm not sure ice streams are the solution to my problem.

Tagged:

Comments

  • xdm
    xdm La Coruña, Spain

    Ice streams are a different beast, they take care of marshaling and unmarshaling of Ice objects and are typically used for dynamic invocation see https://doc.zeroc.com/display/Ice37/JavaScript+Streaming+Interfaces

    Ice for JavaScript currently doesn't provide a mechanism to control data flow comparable to readable streams, we will check if there is something we can do to improve Ice for JavaScript with respect to that.

  • Ok.

    So except from JavaScript, do other languages supported by Ice have a way to control the Ice data flow in a "stream way"? If yes which one (java? c++? ruby?) and where is the related doc?

    Many thanks for your answer.

  • xdm
    xdm La Coruña, Spain

    ObjectAdapter::hold and ObjectAdapter::active allow to pause and resume the dispatch of messages.

    Those are not implemented in JavaScript because browser WebSocket API doesn't provide a mechanism to pause incoming messages. It is probably possible to implement it in NodeJS but we haven't look at it.