Archived

This forum has been archived. Please start a new discussion on GitHub.

Server To Client Callbacks

Hi

I am trying to implement a client-server communication application. The clients may be behind a firewall/NAT, whereas the server is guaranteed to be open. The server needs to make callback function calls to the clients.

Thus the server has to use TCP connections initiated by the client to send callback function requests, but there is no way to directly achieve this in the server using ICE. The only solution currently available is Glacier.

Using Glacier as a separate process which sits in front of the server, seems to be an overkill. Ideally, the server process should be able to do it, since the server itself is not firewalled.

I would like to enhance ICE so that it has the ability to use incoming TCP connections to send function call requests. In fact, for efficiency, the same TCP connection should be usable for both client-to-server function calls, and server-to-client function calls.

How can such a feature be built in ICE? Any pointers on what kind of hooks are available in ICE to add this support would be very helpful

TIA
Regards
K S Sreeram

Comments

  • marc
    marc Florida
    You would have to add an implementation of the Ice::Router interface directly to the Ice core (or to test things, to your server). You could take the Ice::Router implementation of Glacier as an example (but Glacier does more than needed, so you would have to strip it down). Ice::Router is documented in slice/Ice/Router.ice.
  • Hi,

    I am also looking at Server to Client Callbacks, but in another context.

    I am evaluating ICE to use it in an embedded distributed system.
    We want to make server functionality that enables callbacks from a server to several clients. No firewalls are used.
    As we want to do these callbacks selectively, we do not want to use IceStorm.

    I have looked at the example callback, but this callback seems to enable only one client at a time.

    Please correct me if I am mistaken.

    The server seems to use the same TCP/IP connection as was used by the client for the initial request.

    I think only one client at a time is supported because the application object supports only one communicator object.

    Does this imply a memory leak (in the example callback), when several clients are started during one server run?

    To make callbacks to different client I would have to make an adapted application framework to make this work.
    Are there any examples of such an application?

    Thanks in advance

    Albert
  • marc
    marc Florida
    The regular way to do callbacks, is to make your client also a server, and implement Ice objects which the server calls back. This is demonstrated in demo/Ice/callback. In this demo, the client implements CallbackReceiver, and sends a proxy to the server. Using this proxy, the server calls back the client.

    Separate TCP/IP connections are used for incoming and outgoing requests. TCP/IP connections are only shared for both incoming and outgoing requests if "routers" are used, i.e., implementations of Ice::Router, such as Glacier.

    There are no restrictions for the number of clients in the callback demo. There is also no memory leak (why should there be one?).
  • Hi Marc,

    In the example I find two configuration files: config and config.server
    As I understand it in these configuration files the ports to be used by the client and server are specified:
    In both the same ports are specified.
    When I start the server it listens on these ports.
    When the client starts it cannot listen on these ports, so I concluded that the only way the server can reach the client in order to send the callback, is through the connection the client has made.
    When both the client and server are running and I give a (two way) tcp command I still see only one TCP/IP connection.

    I modified the server so it did not send one, but two callbacks to the client.
    This works.

    Later I modified it further. I stored the Current object of the last call and wanted to send the current and previous client each a callback using the two Current objects, when a client calls the server.
    This does not work, the last client gets both callbacks.

    I assume that this is caused by the fact that there is only one communicator object pointer in an application class.
    As a new connection is made the pointer to the old communicator object is replaced by the new one, and the old one 'dangles'.

    Please again, correct me if I am incorrect, or just don't understand how it works :-)

    Albert
  • marc
    marc Florida
    Originally posted by ahartveld
    In the example I find two configuration files: config and config.server
    As I understand it in these configuration files the ports to be used by the client and server are specified:
    In both the same ports are specified.
    When I start the server it listens on these ports.
    When the client starts it cannot listen on these ports, so I concluded that the only way the server can reach the client in order to send the callback, is through the connection the client has made.
    When both the client and server are running and I give a (two way) tcp command I still see only one TCP/IP connection.

    Both configuration files contain the same port, but for different purposes. For the server, this is the port which the server will use to listen for incoming requests. For the client, the port is provided so that the client knows how to reach the server, not to listen on this port. I.e., for the server, the communications endpoints are specified, while for the client a "stringified proxy" is provided, which the client uses to send requests to the server.

    The client endpoints do not have any port, meaning that the client will pick a random port to listen for requests. Since the client sends proxies to the server, this port information can by dynamic, i.e., there is no need for a fixed port for the client.
    Originally posted by ahartveld

    I modified the server so it did not send one, but two callbacks to the client.
    This works.

    Later I modified it further. I stored the Current object of the last call and wanted to send the current and previous client each a callback using the two Current objects, when a client calls the server.
    This does not work, the last client gets both callbacks.

    I assume that this is caused by the fact that there is only one communicator object pointer in an application class.
    As a new connection is made the pointer to the old communicator object is replaced by the new one, and the old one 'dangles'.

    Please again, correct me if I am incorrect, or just don't understand how it works :-)

    Albert

    The Current object has nothing to do with the callback.

    You can run any number of clients for the server. Each client creates a callback object, and sends a proxy to this callback object to the server. The server then uses the callback object to send requests ("callbacks") to the client.

    In this example, the proxies to the callback objects are not stored. But in your own code, you can of course store the callback object proxies. This is typically done in a "subscribe" or "register" method, i.e., the client calls a method on the server to subscribe and send the proxy to its callback object as a parameter. The server then stores all proxies of all subscribers, and calls back on them.
  • Bi-Directional Connections

    Hi

    I was looking into the Router interface and how its being implemented in Glacier.

    What is really needed is the ability to use an already existing incoming connection to dispatch outgoing function calls.

    Using Router to achieve this looks like a round-about way. An additional look-back connection would need to established between the Server and an 'in-process Glacier'. This shouldnt be needed.

    Ideally, a proxy should be associated with an object adapter, so that when a function call needs to be dispatched, the proxy can look at the object adapter, and pick the right existing incoming connection to use for the outgoing function call.

    What would be best approach to implement such a feature.

    Thanks & Regards
    K S Sreeram
  • marc
    marc Florida
    The main problem with bi-directional connections is that the server does not know which incoming connection to use in order to call back a client, given a proxy to some Ice object in that client. This is in particular difficult if NAT firewalls are used, because then the server "sees" an IP address and port which differs from the real client IP address and port.

    Ice::Router solves this problem. It is completely transparent to the server side, and mostly transparent to the client side (except for configuration/setup). I'm afraid I don't have any other solution to offer than Ice::Router.
  • call back example

    Hi Marc.

    Thanks for your answer. I now understand it little better how it works.
    The mechanism as you describe it, suits us well.

    I modified the callback example and now store the proxy of the client by making a copy of it.
    In the next call I sent a callback to both the current and the previous client.
    This works as I want.

    When the server is shutdown I get the following messages:
    Shutting down...
    server: warning: The communicator is not the last Ice object that is
    deleted. (You can disable this warning by setting the
    property `Ice.Warn.Leaks' to 0.)
    Segmentation fault (core dumped)

    How do I cleanup properly (of course I do not want to mask the warning).
    Is the segmentation fault a bug?

    Albert
  • marc
    marc Florida
    The segmentation fault doesn't have anything to do with the warning. Without knowing your code, I cannot tell you where the segmentation fault is coming from.

    The warning only signals that the communicator is not the last object which is destroyed. This is perfectly normal in some cases, for example, if you have a global ObjectAdapterPtr, or other global Ptr's. Ice has a built-in memory leak detector for certain Ptr's, which only works if the destructor of Ice::Communicator is called last. To disable this warning, Ice.Warn.Leaks must be set to 0.
  • Marc,

    The only modification I made was in CallbackI.cpp to the method initiateCallback.
    The rest of the code is unchanged.
    Sorry, but posting seems to discard the formatting.

    void
    CallbackI::initiateCallback(const CallbackReceiverPrx& proxy, const Current& current)
    {
    static CallbackReceiverPrx previousProxy;
    static bool first = true;
    cout << "initiating callback current client" << endl;
    proxy->callback(current.ctx);
    if (!first) {
    cout << "initiating callback to previous client" << endl;
    try {
    previousProxy->callback(current.ctx);
    }
    catch (Ice::ConnectFailedException) {
    // ignore
    cout << "Connect failed exception caught" << endl;
    }
    } else {
    first = false;
    }
    previousProxy = proxy;
    }
  • marc
    marc Florida
    This code is not thread-safe. I suggest to make the static function-scope variables data members of the CallbackI class, and to protect them with a mutex.
  • I agree that the code is not thread safe,
    but that does not explain the segmentation fault.

    It is just an example and I wanted to know if it would work as I thought it would.

    I don't want to spent to much time on this to make it good code,
    but it should not generate an segmentation violation on shutdown.

    This happens even if only one client call is done and then shutdown.
  • mes
    mes California
    Originally posted by ahartveld
    I agree that the code is not thread safe,
    but that does not explain the segmentation fault.

    It is just an example and I wanted to know if it would work as I thought it would.

    I don't want to spent to much time on this to make it good code,
    but it should not generate an segmentation violation on shutdown.

    This happens even if only one client call is done and then shutdown.
    Your server is crashing during destruction of the static proxy, which is attempting to destroy resources that are no longer accessible because the communicator has been destroyed. Specifically, the proxy is attempting to destroy an SSL endpoint, but the IceSSL plug-in was unloaded when the communicator was destroyed, hence the segmentation fault.

    To confirm this diagnosis, you can disable the use of SSL in your configuration files and try the example again. You should still get the leak warning, but not a segmentation fault.

    It is because of situations like this that we advise users to be extremely careful when storing Ice resources in global or static variables.
    Sorry, but posting seems to discard the formatting.
    Check out the formatting tags here. You can use the code tag to show source code in its original format.

    Take care,
    - Mark
  • mes
    mes California
    After looking at this issue further, we discovered that we could make a simple change to the Ice internals that prevents the segmentation fault from occurring in situations like this. The fix will be present in the next Ice release.