Archived

This forum has been archived. Please start a new discussion on GitHub.

ConnectTimeoutException

Hi,
We use php script to call remote ice grid service, and find the following error message:
Ice_UnknownLocalException::__set_state(array( 'unknown' => 'ConnectionI.cpp:1308: Ice::ConnectTimeoutException: timeout while establishing a connection', 'message' => '', 'string' => '', 'code' => 0, 'file' => '/home/okooo/web/data3/Html/passStatTest.php', 'line' => 22, 'trace' => array ( 0 => array ( 'file' => '/home/okooo/web/data3/Html/passStatTest.php', 'line' => 22, 'function' => 'computepassstatbyid', 'class' => 'Ice_ObjectPrx', 'type' => '->', 'args' => array ( 0 => '7822199', 1 => 'WCFourGoal', ), ), ), ))
will anybody know how to resolve this problem? And here is my network tracing:
[ 08/24/09 12:00:57.250 Locator: searching for adapter by id
  adapter = PsServer192.168.8.30.PsServerAdapter ]
[ 08/24/09 12:00:57.251 Network: trying to establish tcp connection to 192.168.8.36:4061 ]
[ 08/24/09 12:00:57.253 Network: tcp connection established
  local address = 192.168.8.31:40136
  remote address = 192.168.8.36:4061 ]
[ 08/24/09 12:00:57.254 Network: received 14 of 14 bytes via tcp
  local address = 192.168.8.31:40136
  remote address = 192.168.8.36:4061 ]
[ 08/24/09 12:00:57.254 Protocol: received validate connection
  message type = 3 (validate connection)
  compression status = 0 (not compressed; do not compress response, if any)
  message size = 14 ]
[ 08/24/09 12:00:57.254 Protocol: sending asynchronous request
  message type = 0 (request)
  compression status = 0 (not compressed; do not compress response, if any)
  message size = 98
  request id = 1
  identity = OkoooGrid/Locator
  facet =
  operation = findAdapterById
  mode = 1 (nonmutating)
  context =  ]
[ 08/24/09 12:00:57.254 Network: sent 98 of 98 bytes via tcp
  local address = 192.168.8.31:40136
  remote address = 192.168.8.36:4061 ]
[ 08/24/09 12:00:57.255 Network: received 14 of 14 bytes via tcp
  local address = 192.168.8.31:40136
  remote address = 192.168.8.36:4061 ]
[ 08/24/09 12:00:57.255 Network: received 82 of 82 bytes via tcp
  local address = 192.168.8.31:40136
  remote address = 192.168.8.36:4061 ]
[ 08/24/09 12:00:57.255 Protocol: received reply
  message type = 2 (reply)
  compression status = 0 (not compressed; do not compress response, if any)
  message size = 96
  request id = 1
  reply status = 0 (ok) ]
[ 08/24/09 12:00:57.255 Locator: retrieved endpoints from locator, adding to locator table
  adapter = PsServer192.168.8.30.PsServerAdapter
  endpoints = tcp -h 192.168.8.30 -p 57732:tcp -h 58.83.184.30 -p 57732: ]
[ 08/24/09 12:00:57.255 Network: trying to establish tcp connection to 192.168.8.30:57732 ]
[ 08/24/09 12:00:57.256 Network: tcp connection established
local address = 192.168.8.31:54076
  remote address = 192.168.8.30:57732 ]
[ 08/24/09 12:01:02.257 Network: closing tcp connection
  local address = 192.168.8.31:54076
  remote address = 192.168.8.30:57732 ]
[ 08/24/09 12:01:02.265 Retry: connection to endpoint failed and no more endpoints to try
  ConnectionI.cpp:1308: Ice::ConnectTimeoutException:
  timeout while establishing a connection ]
[ 08/24/09 12:01:02.265 Network: trying to establish tcp connection to 58.83.184.30:57732 ]
[ 08/24/09 12:01:02.268 Network: tcp connection established
  local address = 58.83.184.31:40543
  remote address = 58.83.184.30:57732 ]
[ 08/24/09 12:01:07.270 Network: closing tcp connection
  local address = 58.83.184.31:40543
  remote address = 58.83.184.30:57732 ]
[ 08/24/09 12:01:07.283 Retry: connection to endpoint failed, trying next endpoint
  ConnectionI.cpp:1308: Ice::ConnectTimeoutException:
  timeout while establishing a connection ]
[ 08/24/09 12:01:07.283 Locator: removed endpoints from locator table
  adapter = PsServer192.168.8.30.PsServerAdapter
  endpoints = tcp -h 192.168.8.30 -p 57732:tcp -h 58.83.184.30 -p 57732: ]
[ 08/24/09 12:01:07.283 Retry: retrying operation call because of exception
  ConnectionI.cpp:1308: Ice::ConnectTimeoutException:
  timeout while establishing a connection ]
[ 08/24/09 12:01:07.283 Locator: searching for adapter by id
  adapter = PsServer192.168.8.30.PsServerAdapter ]
[ 08/24/09 12:01:07.283 Protocol: sending asynchronous request
  message type = 0 (request)
  compression status = 0 (not compressed; do not compress response, if any)
  message size = 98
  request id = 2
  identity = OkoooGrid/Locator
  facet =
  operation = findAdapterById
  mode = 1 (nonmutating)
  context =  ]
[ 08/24/09 12:01:07.283 Network: sent 98 of 98 bytes via tcp
  local address = 192.168.8.31:40136
  remote address = 192.168.8.36:4061 ]
[ 08/24/09 12:01:07.284 Network: received 14 of 14 bytes via tcp
  local address = 192.168.8.31:40136
  remote address = 192.168.8.36:4061 ]
[ 08/24/09 12:01:07.284 Network: received 82 of 82 bytes via tcp
  local address = 192.168.8.31:40136
  remote address = 192.168.8.36:4061 ]
[ 08/24/09 12:01:07.284 Protocol: received reply
  message type = 2 (reply)
  compression status = 0 (not compressed; do not compress response, if any)
  message size = 96
  request id = 2
  reply status = 0 (ok) ]
[ 08/24/09 12:01:07.284 Locator: retrieved endpoints from locator, adding to locator table
  adapter = PsServer192.168.8.30.PsServerAdapter
  endpoints = tcp -h 192.168.8.30 -p 57732:tcp -h 58.83.184.30 -p 57732: ]
[ 08/24/09 12:01:07.285 Network: trying to establish tcp connection to 192.168.8.30:57732 ]
[ 08/24/09 12:01:07.285 Network: tcp connection established
  local address = 192.168.8.31:54080
  remote address = 192.168.8.30:57732 ]
[ 08/24/09 12:01:12.287 Network: closing tcp connection
  local address = 192.168.8.31:54080
  remote address = 192.168.8.30:57732 ]
[ 08/24/09 12:01:12.287 Retry: connection to endpoint failed and no more endpoints to try
  ConnectionI.cpp:1308: Ice::ConnectTimeoutException:
  timeout while establishing a connection ]
[ 08/24/09 12:01:12.287 Network: trying to establish tcp connection to 58.83.184.30:57732 ]
[ 08/24/09 12:01:12.288 Network: tcp connection established
  local address = 58.83.184.31:40547
  remote address = 58.83.184.30:57732 ]
[ 08/24/09 12:01:17.289 Network: closing tcp connection
  local address = 58.83.184.31:40547
  remote address = 58.83.184.30:57732 ]
[ 08/24/09 12:01:17.289 Retry: connection to endpoint failed, trying next endpoint
  ConnectionI.cpp:1308: Ice::ConnectTimeoutException:
  timeout while establishing a connection ]
[ 08/24/09 12:01:17.289 Locator: removed endpoints from locator table
  adapter = PsServer192.168.8.30.PsServerAdapter
  endpoints = tcp -h 192.168.8.30 -p 57732:tcp -h 58.83.184.30 -p 57732: ]
[ 08/24/09 12:01:17.290 Retry: cannot retry operation call because retry limit has been exceeded
  ConnectionI.cpp:1308: Ice::ConnectTimeoutException:
  timeout while establishing a connection ]
[ 08/24/09 12:01:17.290 Protocol: sending close connection
  message type = 4 (close connection)
 08/24/09 12:01:17.290 Protocol: sending close connection
  message type = 4 (close connection)
  compression status = 1 (not compressed; compress response, if any)
  message size = 14 ]
[ 08/24/09 12:01:17.290 Network: sent 14 of 14 bytes via tcp
  local address = 192.168.8.31:40136
  remote address = 192.168.8.36:4061 ]
[ 08/24/09 12:01:17.290 Network: closing tcp connection
  local address = 192.168.8.31:40136
  remote address = 192.168.8.36:4061 ]

Comments

  • mes
    mes California
    Hi,

    The trace information indicates that the client is able to open a socket connection to the server at both of its endpoints (192.168.8.30/57732 and 58.83.184.30/57732). Once an Ice client successfully opens a socket connection, it waits for the server to send an initial handshake message, after which the Ice connection is considered to be fully established.

    It appears your client is using a 5s timeout and that the client is not receiving the server's initial handshake message within this timeout period, therefore it reports a ConnectTimeoutException. The most likely reason for a server to fail to send this handshake message is thread starvation, that is, no more threads are available in the server-side thread pool to accept the new connection and send the message. In this situation we generally recommend attaching a debugger to your server to examine the state of all of its threads. Is it possible that your server's thread pool threads could all be busy?

    If you're still having trouble, please specify the versions of Ice and your operating system and compiler.

    Regards,
    Mark