Archived

This forum has been archived. Please start a new discussion on GitHub.

Help! Why ice_response for AMD block?

Hi you guru,

Here is my problem. I have a client and a server, the client uses AMI, and make many calls, the server handle those calls in AMD ways, cause it takes long time(e.x. 2 minutes or more). Server has a independent thread pool to handle the response (the size of thread pool is constant, like 1 or 50). Clients do heartbeat every some seconds, and if it find it cannot connect to server, it call connection->close(true) to forcely close that connection ( got from cache ). Now Problem is:

I send a lot of calls from client, and server got those calls, put them in a process queue and return (The AMD way). After processing, server create a response object, and put that object in another response queue, the response thread (from response thread pool) picks it and call ice_response() of the callback object (from AMD call interface). Everything works fine, But when I test disconnecting, problem occurrs, sometimes the call to ice_response() is ok, but sometimes the call to ice_response() blocked! (long time , maybe 10-30 minutes). And because my response thread pool size is constant, so it ran out quickly, the server became no response!

Does there any suggestion on why ice_response() for AMD might block? I cannot set the timeout for this call, cause it might took a long time to finish one call, and the time cannot be told. I didn't override any properties except Ice.Override.ConnectTimeout = 1000 (1 sec), no override for Ice.ACM.Client, Ice.ACM.Server, Ice.MonitorConnections.

Comments

  • I didn't set Ice.Override.Timeout because the time for processing a call is uncertain, and I did control of it by heartbeat and call to connection->close().

    I want ice_response() for AMD callbacker to be non-block operation, is this possible?

    Regards
    Xiang.
  • the block only happened after I disconnect the network cable while invoking many AMI calls.
  • soloman817 wrote: »
    I didn't set Ice.Override.Timeout because the time for processing a call is uncertain, and I did control of it by heartbeat and call to connection->close().

    I want ice_response() for AMD callbacker to be non-block operation, is this possible?

    Regards
    Xiang.

    No, ice_response() will block until the message is either sent (i.e., accepted by the TCP/IP stack), an exception occurs, or a timeout kicks in (if you have a timeout set).

    In many cases, you won't get an exception if the network cable is unplugged (this is nothing specific to Ice, but applies to all TCP/IP based applications). Try to enable network tracing on the server side to find out more (Ice.Trace.Network=3).
  • Thanks alot

    Let me get more Trace information :-)
  • well, I cannot get information from the trace, too much, for I'm testing the stress. and all those trace message is saying: sent n bytes, recieve n bytes, etc.

    well, what I think is I got the AMD_Callback Ptr object, can I set a timeout before I call its ice_response() ? I mean the timeout only for send response, I don't want to set the timeout the client call server, but is there any way to set the timeout to a small time when sending back the response?
  • well, I've tried this method:

    when I got the call, I record the Current.con and callback object ptr, and when the server find the client is dead (by heartbeat check in another thread), I call con->close(true), but it cannot produce exception, the thread calling ice_response still block.
  • If you forcefully close the connection, the thread trying to send a response should return immediately with an exception. What does the trace look like at the point in time when you call close(true)? The best way for us to help you is if you provide a small self-contained example that demonstrates the problem.
  • By the way, which version of Ice are you using? Please make sure to test your application with the latest version of Ice.
  • bernard
    bernard Jupiter, FL
    soloman817 wrote: »
    well, what I think is I got the AMD_Callback Ptr object, can I set a timeout before I call its ice_response() ? I mean the timeout only for send response, I don't want to set the timeout the client call server, but is there any way to set the timeout to a small time when sending back the response?

    Hi Zhang Xiang,

    You set the timeout for responses through the .Endpoints property of your object adapter, e.g.

    # 1 second
    MyOA.Endpoints=tcp -h myhost -p 10000 -t 1000

    If you don't want this timeout to be inserted in the proxies created by this object adapter, you should override it with PublishedEndpoints, e.g.

    # 60 seconds
    MyOA.PublishedEndpoints=tcp -h myhost -p 10000 -t 60000

    Cheers,
    Bernard