Async bidirectional issue

Kruppy

Hello,
in our architecture client is using bidir async mode to call the server methods in long-running operations, so once response is ready, client's is being called with "response's ready".
Bidir was choosen to bypass firewall. Well, it seems it's not working all the time. Sometimes works, sometimes doesn't. Looks like networking issues. What we might have done wrong? How can we make solution portable?
Any insights will be very much appreciated.

marc

I'm afraid from your description, it's impossible to say what's going wrong. You must provide a lot more details about the problems. Also let us know your Ice version, which language mapping you are using, your platform, and compiler. Please include a very small, but self-contained example that demonstrates the problem.

Kruppy

Marc,
Ice version 2.1.2(sorry, haven't migrated yet),
language mapping: Client- slice2cpp, Server - slice2java,
platform: client runs on windows 2000/XP, server - windows or Linux,
compiler: VC71

As an example, we have used demos/Ice/bidir. CallbackSender and CallbackReciver interfaces are implemented on client/server sides accordingly, and adapter is set on the proxy.
The client is able to submit query all the time, no exceptions thrown; when server's trying to make a call back with response, it seems client is not listening, but again, no exceptions thrown. ( client proxy times out or something?)

We have identified one use-case though. If Check Point Network Security Software is installed on the pc, Point Network SecuPoint must be UN-checked (LAN connection properties) in order to get bidir working.

benoit

Hi,

I'm afraid there's still too little information to figure out what could be wrong. Could you reproduce the issue and run the client/server with the command line options --Ice.Trace.Network=2 and --Ice.Trace.Protocol? You should see the client and server requests/responses as well as the network connection establishment/closure with these trace properties enabled. Please post the traces here, this will hopefully give us some clues on your issue!

Cheers,
Benoit.

Kruppy

I did enable tracing.
I will try reproduce the issue.
In the meanwhile, on the client (MFC based Excel Add-in) I have the following exceptions (it seems it does not effect bidir) in debug output:
...
User yuri attempts to login 10/24/2006, 12:58:49
First-chance exception at 0x7c812a5b in EXCEL.EXE: Microsoft C++ exception: Ice::TimeoutException @ 0x045ae344.
Authentication succeded for user yuri, 10/24/2006, 12:58:49
The thread 'Win32 Thread' (0x1470) has exited with code 0 (0x0).
First-chance exception at 0x7c812a5b in EXCEL.EXE: Microsoft C++ exception: Ice::ConnectionLostException @ 0x045ae360.
...

This Ice::TimeoutException is printed out just after the very first remote call, between valid custom trace info confirming user has loged in. Now I am trying to zero in on what may be causing these exeptions. Any suggestions?

matthew

A TimeoutException indicates that the client did not receive a reply to its invocation within the specified timeout period. If you don't expect the network to be slow (for example, you are only talking within a fast LAN) then quite often this is indicative of a deadlock, or the server running out of threads.

Kruppy

Matthew,
the following is the code sample which indicates the moment where mentioned Ice::TimeoutException is printed out in debug output (just after succesfull remote invocation)
metaPrx->login( userID, password )
Call returns quickly, so there is no deadlock or something..
....
//logging
TRACE("User %s attempts to login %s\n", m_strUserID, CTime::GetCurrentTime().Format((UINT)IDS_DUMP_TIME_FORMAT_STR) );
//do login
ServerAccessPoint& gapInstance = ServerAccessPoint::Instance();
ServicePrx metaPrx = gapInstance.GetMetaProxy();
LoginPtr loginPtr = metaPrx->login( userID, password );
if ( loginPtr )
{
if( loginPtr->folder.size() )
{
//SUCCESS
TRACE("Authentication succeded for user %s, %s\n", m_strUserID, CTime::GetCurrentTime().Format((UINT)IDS_DUMP_TIME_FORMAT_STR) );
}
}
...

benoit

Hi,

I wouldn't worry about these first chance exception debug messages. It just means that an Ice::TimeoutException was raised in the Ice core. It's catched by the Ice internals -- that's why you don't "see" it in your code.

See also this KB article or this explanation for more information on first chance exceptions.

Cheers,
Benoit.

Kruppy

Bidirectional connection timing out

We have been migrated to the ICE v3.2 finally.
Using bidirectional callback, we are still having the same issue with firewall (VPN) described earlier.
But for now I want to resolve a different one. Client can submit a long running request and able to cancel it. But it seems connection is timing out after a minute or so if it stays idle, even though the server proxy is explicitly created with ice_timeout(-1).
The code sample follows:
m_pCallBackClientIPtr = new CallBackClientI(m_QueryQueue, &m_runThreadInfo, this->m_hWnd);
m_pBiDirAdapterPtr = m_pIceCommunicator->createObjectAdapter("");
//activate servant
m_pBiDirAdapterPtr->add(m_pCallBackClientIPtr, m_SessionIdentity);
//Activate endpoint that belong to this object adapter
m_pBiDirAdapterPtr->activate();
m_BiDirCallBackQuerySrvcPrx->ice_getConnection()->setAdapter(m_pBiDirAdapterPtr);

After a minute or so consecutive usage of proxy m_BiDirCallBackQuerySrvcPrx fails with exception thrown within .cpp file which was slice-generated from .ice interface.

Please advise what's could be wrong here
Thank you

marc

Do you create the client proxy with no timeout? The client opens the bi-dir connection and controls its lifecycle, not the server. Switch on network tracing (Ice.Trace.Network=2) in both the client and the server to find out what is closing the connection.

Kruppy

Yes, the proxy is created with ice_timeout(-1).
To be more specific about exception, __del->addClientExecute(ident, reqlst, proxy, __ctx) call throwns an exception of type LocalExceptionWrapper as illustraded bellow, so server is not envolved here.

::IceInternal::Handle< ::IceDelegate::Ice::Object> __delBase;
try
{
__checkTwowayOnly(__com__xpheria__queryservice__CallbackSender__addClientExecute_name);
__delBase = __getDelegate();
::IceDelegate::com::xpheria::queryservice::CallbackSender* __del = dynamic_cast< ::IceDelegate::com::xpheria::queryservice::CallbackSender*>(__delBase.get());
__del->addClientExecute(ident, reqlst, proxy, __ctx);
return;
}
catch(const ::IceInternal::LocalExceptionWrapper& __ex)
{
__handleExceptionWrapper(__delBase, __ex);
}

It seems getDelegate() fails. What does it mean?

marc

We really cannot explain Ice internals here in these forums. The easiest solution would be to provide a small, self-contained example that demonstrates the problem.

benoit

Btw, the exception is most likely expected if the connection is closed. The first thing to do would be to determine and tell us which exception is thrown. You can easily do this by adding a try catch block around the proxy invocation to print the exception... (no need to dig into the Ice internals ). For example:

try
{
   m_BiDirCallBackQuerySrvcPrx->addClientExecute(...);
}
catch(const Ice::LocalExcepiton& ex)
{
   cerr << ex << endl;
}

Also, make sure to disable activation connection management in the client with Ice.ACM.Client=0 when using bidirectional connection, otherwise the connection will be closed if there's no activity on the connection (the default is 60s). See here in the Ice manual for more details on bidirectional connections.

Cheers,
Benoit.

Kruppy

Benoit, thanks for ACM hint.
As it appeared, I have misssed that specific part somehow.
But do I really need to disable ACM? Can i say Ice.ACM.Client=120 (s) and even make it configurable on the client side? Do I need to override Ice.ACM.Client property on both sides - client and sever (meaning server becomes a client when it's making callback).
Thank you agian

benoit

You should disable ACM otherwise the client might close the bidirectional connection and the server won't be able to callback your client over that connection. You only need to disable it in the client, by default ACM is disalbed on the server side. See here for more information on ACM.

Cheers,
Benoit.

Kruppy

Thank you, Benoit, it was very helpful