Archived

This forum has been archived. Please start a new discussion on GitHub.

Ice Grid Endpoint Issues

I am having a problem with ice grid. The Ice server runs on a computer with multiple network cards, and by default the ice grid lists both of the ip address's as endpoints. This works but can be slow depending on which endpoint the client tries first, as only one of the networks is reachable by the client. To work around this i wanted to use the host name as the endpoint. but this does not work. The client get an ice connection refused exception. The network trace shows it is using the right ip address and port, and wire shark on both ends shows the packets but it never connects.


works but slow
id.properties->setProperty("GigeAdapter.Endpoints","tcp");
doesn't work
id.properties->setProperty("GigeAdapter.Endpoints","tcp -h " + string(hostName));

client IP= 172.20.35.39
iceGrid IP=172.20.1.184
server IP=172.20.37.117

Client trace out put
-- 02/25/12 17:10:59.210 Network: trying to establish tcp connection to 172.20.1
.84:11010
-- 02/25/12 17:10:59.210 Network: tcp connection established
   local address = 172.20.35.39:3838
   remote address = 172.20.1.84:11010
-- 02/25/12 17:10:59.210 Protocol: received validate connection
   message type = 3 (validate connection)
   compression status = 0 (not compressed; do not compress response, if any)
   message size = 14
-- 02/25/12 17:10:59.225 Protocol: sending asynchronous request
   message type = 0 (request)
   compression status = 0 (not compressed; do not compress response, if any)
   message size = 64
   request id = 1
   identity = IceGrid/Locator
   facet =
   operation = findAdapterById
   mode = 1 (nonmutating)
   context =
-- 02/25/12 17:10:59.225 Protocol: received reply
   message type = 2 (reply)
   compression status = 0 (not compressed; do not compress response, if any)
   message size = 62
   request id = 1
   reply status = 0 (ok)
-- 02/25/12 17:10:59.241 Network: trying to establish tcp connection to 172.20.3
7.117:61949
-- 02/25/12 17:11:00.163 Network: failed to establish tcp connection
   local address: 0.0.0.0:3839
   remote address: 172.20.37.117:61949
   Network.cpp:1387: Ice::ConnectionRefusedException:
   connection refused: The remote system refused the network connection.
-- 02/25/12 17:11:00.163 Retry: retrying operation call because of exception
   Network.cpp:1387: Ice::ConnectionRefusedException:
   connection refused: The remote system refused the network connection.
-- 02/25/12 17:11:00.178 Protocol: sending asynchronous request
   message type = 0 (request)
   compression status = 0 (not compressed; do not compress response, if any)
   message size = 64
   request id = 2
   identity = IceGrid/Locator
   facet =
   operation = findAdapterById
   mode = 1 (nonmutating)
   context =
-- 02/25/12 17:11:00.178 Protocol: received reply
   message type = 2 (reply)
   compression status = 0 (not compressed; do not compress response, if any)
   message size = 62
   request id = 2
   reply status = 0 (ok)
-- 02/25/12 17:11:00.178 Network: trying to establish tcp connection to 172.20.3
7.117:61949
-- 02/25/12 17:11:01.257 Network: failed to establish tcp connection
   local address: 0.0.0.0:3840
   remote address: 172.20.37.117:61949
   Network.cpp:1387: Ice::ConnectionRefusedException:
   connection refused: The remote system refused the network connection.
-- 02/25/12 17:11:01.272 Retry: cannot retry operation call because retry limit
has been exceeded
   Network.cpp:1387: Ice::ConnectionRefusedException:
   connection refused: The remote system refused the network connection.
Network.cpp:1387: Ice::ConnectionRefusedException:
connection refused: The remote system refused the network connection.


Server Trace output
config1.txt
-! 02/25/12 17:18:14.674 warning: deprecated property: Ice.Trace.Location
-- 02/25/12 17:18:15.249 Network: attempting to bind to tcp socket 192.168.32.13
6:0
-- 02/25/12 17:18:15.290 Network: accepting tcp connections at 192.168.32.136:63
215
-- 02/25/12 17:18:15.316 Network: published endpoints for object adapter `GigeAd
apter':
   tcp -h junelake -p 61949
-- 02/25/12 17:18:15.390 Network: trying to establish tcp connection to 172.20.1
.84:11010
-- 02/25/12 17:18:15.399 Network: tcp connection established
   local address = 172.20.37.117:63218
   remote address = 172.20.1.84:11010
-- 02/25/12 17:18:15.407 Network: trying to establish tcp connection to 172.20.1
.84:48196
-- 02/25/12 17:18:15.411 Network: tcp connection established
   local address = 172.20.37.117:63219
   remote address = 172.20.1.84:48196
Server started.

Comments

  • benoit
    benoit Rennes, France
    Hi,

    It looks like a configuration issue on the server side:
    Network: accepting tcp connections at 192.168.32.136:63215
    Network: published endpoints for object adapter `GigeAdapter':
       tcp -h junelake -p 61949
    

    The server is publishing the right endpoint but it's not listening on the IP address you are expecting. Is the "junelake" hostname correctly resolving to 172.20.37.117 on the machine which is running the server?

    Could you provide the configuration file for the server or the IceGrid XML deployment descriptor if you use IceGrid deployments?

    Cheers,
    Benoit.
  • You are right, on the server, junelake is not resolving to the 172.20 ip address but instead the 192.168. Is there a way I can have it listen on both ip addresses, but still publish the hostname to the ice grid server.

    Here is my complete server configuration.

    id.properties = Ice::createProperties();
    id.properties->setProperty("Ice.MessageSizeMax","10240");
    id.properties->setProperty("GigeAdapter.AdapterId",node);
    id.properties->setProperty("GigeAdapter.Endpoints","tcp -h " + string(hostName));
    id.properties->setProperty("Ice.Default.Locator","IceGrid/Locator:default -h olive -p 11010");
    id.properties->setProperty("Ice.Warn.UnknownProperties","0");
    id.properties->setProperty("Ice.Trace.Location","0");
    id.properties->setProperty("Ice.Trace.Network","0");
    id.properties->setProperty("Ice.Trace.Protocol","0");
    id.properties->setProperty("Ice.Trace.Retry","0");
    id.properties->setProperty("IceGrid.Registry.Trace.Replica","0");
    communicator = Ice::initialize(id);
    
  • benoit
    benoit Rennes, France
    I assume that you don't use IceGrid XML descriptors to configure the server. You can set the <adapter name>.PublishedEndpoints property to configure the endpoints that will be published in Ice proxies (including the one published to IceGrid) and if you need those endpoints to be different from the physical endpoints. In your case, you could use:
    id.properties->setProperty("GigeAdapter.Endpoints","tcp"); // no -h means listen on all interfaces
    id.properties->setProperty("GigeAdapter.PublishedEndpoints","tcp -h " + string(hostName));
    

    For more information on object adapter endpoints see this section in the Ice manual.

    Cheers,
    Benoit.
  • That works slightly better but always ends up with port 0. Is there a way to specify automatic port selection, because I would like to have more then one server per computer.
  • benoit
    benoit Rennes, France
    Hi,

    Sorry I forgot to mention this, you indeed need to use a fixed port when using published endpoints. It's currently not possible to use system allocated ports with published endpoints. You'll need to configure different ports to run several server on the same machine.

    Another option is to programmatically compute the published endpoints property in the server after the object adapter creation, here's some sample code which obtains the physical endpoints used by the object adapter and creates a published endpoint property containing the physical endpoint matching a given host name:
       Ice::ObjectAdapterPtr adapter = communicator()->createObjectAdapter("Hello");
    
        string publishedEndpoints;
        Ice::EndpointSeq physicalEndpoints = adapter->getEndpoints();
        string hostName = "junelake"; // Obtain from configuration
        for(Ice::EndpointSeq::const_iterator q = physicalEndpoints.begin(); q != physicalEndpoints.end(); ++q)
        {
            Ice::IPEndpointInfoPtr physical = Ice::IPEndpointInfoPtr::dynamicCast((*q)->getInfo());
            if(physical && physical->host == hostName)
            {
                publishedEndpoints = (*q)->toString();
                break;
            }
        }
        if(!publishedEndpoints.empty())
        {
            // Set the new published endpoints and notify the adapter that the published endpoints changed.
            communicator()->getProperties()->setProperty("Hello.PublishedEndpoints", publishedEndpoints);
            adapter->refreshPublishedEndpoints();
        }
    
        adapter->activate();
    

    Of course, the simplest would be to listen on a single interface, you wouldn't need to use the published endpoints property in this case. Do you really need to listen on this other interface which isn't accessible by all Ice clients?

    Cheers,
    Benoit.
  • I don't need both to work, but I assumed it would be easier then deciding which one was the right one.