LAN/WAN problems

grayclock · September 2007

I have the following ICE objects:

dictionary<string, string> ProfileDetails;

class Player {
............
............
void setProfile(ProfileDetails newProfile);
ProfileDetails getProfile();

string name;
ProfileDetails profile;
};

interface PlayerManager
{
Player* add(ProfileDetails profile) throws PlayerExistExceptions;
Player* find(string username);
};

Here's how it works:

PlayerManager::find

PlayerPrx PlayerManagerI::find(const ::std::string& name, const ::Ice::Current& c )
{
IceUtil::Mutex::Lock lock(*this);
int count = _index->count(name);
if (count ==0) return 0;
vector<Ice::Identity> idents = _index->find(name);
return PlayerPrx::uncheckedCast(c.adapter->createProxy(idents[0]));
};

This is the problem: When the client and the server are both in the local network (LAN), everything works fine. The method "find" finds and creates the correct object, which I use after that. And example of the code for the client:

PlayerPrx prx = playerManager->find(username);
if (prx)
{
ProfileDetails details = prx->getProfile();
.....................
.....................
....................
}

When the client and the server are not on the same LAN (i.e. across the internet), this line...

ProfileDetails details = prx->getProfile();

...after a certain amount of time, generates an error for the client:

External exception EEFFACE.

...and an error for the server:

[ 09/13/07 23:06: 50.531 warning: connection exception:
TcpTransceiver.cpp:297: Ice::ConnectionLostException: Connection lost: WSAECONNRESET
Local address = xxx.xxx.xxx.xxxx
Remote address = yyy.yyy.yyy ]
[ 09/13/07 23:06:50.531 Network: shutting down tcp connection for reading and writing
Local address = xxx.xxx.xxx.xxxx
Remote address = yyy.yyy.yyy ]
[ 09/13/07 23:06:50.531 Network: closing tcp connection
Local address = xxx.xxx.xxx.xxx
Remote address = yyy.yyy.yyy ]

I'm using this software: CodeGear RAD 2007 C++ Builder & Ice 3.2.1

marc · September 2007

I'm afraid I don't know what EEFACE is. It is nothing related to Ice. You should consult your compiler vendor for more information.

The warning you see on the server simply indicates that your client has crashed, and thus the connection has been reset.

grayclock · September 2007

We tried using a variety of computers for the server and the client, and the problem happens over WAN every time, and never over LAN.

marc · September 2007

I'm afraid I can't offer more advice, other than that you should try to find out what the EEFACE exception means, and where it comes from. Also, try the "hello world" demo over the WAN.

Note that C++ Builder 2007 is not one of our supported compilers.

grayclock · September 2007

Thanks,
Bojko

matthew · September 2007

According to this article Translate C++ Exceptions to VCL Exceptions

If you allow a non-VCL/CLX exception to leak from an event handler in your code back into the VCL/CLX framework, the framework will display a rather useless message, something along the lines of "External Exception EEFFACE". Essentially, this message means that a C++ exception has leaked into the framework and the framework doesn't know what to do.

So it sounds like Ice invocation is failing and throwing an exception which you should catch and handle...

So you should instead do something like:

try
{
    ProfileDetails details = prx->getProfile();
}
catch(const Ice::Exception& e)
{
    // handle the error.
    cerr << "getProfile failed: " << e << endl;
}

Of course you may need to do more appropriate error handling.

grayclock · September 2007

I tried this with C++ Builder 2006, and caught the exception.
It's :
IceUtil::Exception
File :Network.cpp
Line: 670
Error: 10060

stack dump:

:006FE9E4 IceInternal::doConnect(fd=3708, addr=:01449AD8, timeout=-1)
:00767D05 IceInternal::TcpConnector::connect(this=:01449AC0, timeout=-1)
:00673218 IceInternal::OutgoingConnectionFactory::create(this=:013C0C10, endpts=:0012F80C, moreEndpts=false, threadPerConnection=false, compress=:014665AC)
:00743A53 IceInternal::RoutableReference::createConnection(this=:01B267B0, allEndpoints=:0012F8A4, comp=:014665AC)
:00745266 IceInternal::DirectReference::getConnection(this=:01B267B0, comp=:014665AC)
:00735C96 IceDelegateM::Ice::Object::setup(this=:0146659C, ref=:01457C4C)
:007347EF IceProxy::Ice::Object::__getDelegate(this=:01457C28)

marc · September 2007

Print the exception to get more details about what's going wrong.

grayclock · September 2007

result:

getProfile failed: Network.cpp:670: Ice::ConnectFailedException:
connect failed: WSAETIMEDOUT

source:

int main(int argc, char* argv[])
{
   PlayerManagerPrx playerManager;
   Ice::CommunicatorPtr communicator;
   PlayerPrx CurrentPlayer;

   communicator = Ice::initialize();

   Ice::ObjectPrx base =  communicator->stringToProxy("PlayerManager:tcp -h xxx.xxx.xxx.xxx -p 10003");

   playerManager = PlayerManagerPrx::checkedCast(base);
   //---------------------------------------------------
   CurrentPlayer = playerManager->login("1","1");
   try
   {
        CurrentPlayer->getProfile();
    }
    catch(const Ice::Exception& e)
    {
	cout << "getProfile failed: " << e << endl;
    }
    return 0;
}

marc · September 2007

This exception indicates that your client tries to establish a connection, but the attempt to establish the connection times out. Try to enable network tracing to see what connections Ice tries to open and close. Set the property Ice.Trace.Network=2. Also, see this FAQ, which does not directly refer to connection establishment timeouts, but provides a general overview for how to debug such problems.

Note that in most cases, such a timeout is caused by a firewall, i.e., the firewall blocks the connection to the specified host/port. This would also explain why you see the problem on the WAN only, but not on the LAN.

grayclock · September 2007

The firewall doesn't appear to be the problem. I have made sure all firewalls are turned off, and ports are properly forwarded over the WAN connection.

I'm using the Playermanager object, and after the problem happens I can still continue to use Playermanager. The connection is lost only when I try to use the result of Playermanager->Find()/login()

How can it be the firewall if the connection is lost ONLY when I use the server-side created object? Any other ideas on what could cause this problem?

Server log:

[ 09/14/07 15:25:40.875 Network: attempting to bind to tcp socket xxxx.xxxx.xxx.xxx:10003 ]
[ 09/14/07 15:25:40.890 Network: accepting tcp connections at xxxx.xxxx.xxx.xxx:10003 ]
[ 09/14/07 15:25:40.890 Network: attempting to bind to tcp socket 127.0.0.1:10003 ]
[ 09/14/07 15:25:40.890 Network: accepting tcp connections at 127.0.0.1:10003 ]
[ 09/14/07 15:25:40.890 Freeze.DbEnv: opening database environment "db" ]
[ 09/14/07 15:25:45.187 Network: accepted tcp connection
  local address = xxxx.xxxx.xxx.xxx:10003
  remote address = yyy.yyy.yyy.yyy:2182 ]
[ 09/14/07 15:25:45.187 Protocol: sending validate connection
  message type = 3 (validate connection)
  compression status = 0 (not compressed; do not compress response, if any)
  message size = 14 ]
[ 09/14/07 15:25:45.203 Protocol: received request
  message type = 0 (request)
  compression status = 0 (not compressed; do not compress response, if any)
  message size = 80
  request id = 1
  identity = PlayerManager
  facet =
  operation = ice_isA
  mode = 1 (nonmutating)
  context =  ]
[ 09/14/07 15:25:45.203 Protocol: sending reply
  message type = 2 (reply)
  compression status = 0 (not compressed; do not compress response, if any)
  message size = 26
  request id = 1
  reply status = 0 (ok) ]
[ 09/14/07 15:25:48.078 Protocol: received request
  message type = 0 (request)
  compression status = 0 (not compressed; do not compress response, if any)
  message size = 49
  request id = 2
  identity = PlayerManager
  facet =
  operation = find
  mode = 2 (idempotent)
  context =  ]
[ 09/14/07 15:25:48.078 Protocol: sending reply
  message type = 2 (reply)
  compression status = 0 (not compressed; do not compress response, if any)
  message size = 27
  request id = 2
  reply status = 0 (ok) ]
[ 09/14/07 15:25:53.000 warning: connection exception:
TcpTransceiver.cpp:297: Ice::ConnectionLostException:
connection lost: WSAECONNRESET
local address = xxxx.xxxx.xxx.xxx:10003
remote address = yyy.yyy.yyy.yyy:2182
[ 09/14/07 15:25:53.000 Network: shutting down tcp connection for reading and wr
iting
  local address = xxxx.xxxx.xxx.xxx:10003
  remote address = yyy.yyy.yyy.yyy:2182 ]
[ 09/14/07 15:25:53.000 Network: closing tcp connection
  local address = xxxx.xxxx.xxx.xxx:10003
  remote address = yyy.yyy.yyy.yyy:2182 ]

marc · September 2007

The server log only indicates that the client drops the connection, which is expected, since the client does not properly destroy the communicator in your test application. You must look at the client log to find out what's going on, since this is where the error occurs.

grayclock · September 2007

I am sorry marc,
but your advice sounds absolutely incompetent.
The same server and client was running on VS C/C++ without any problems.

marc · September 2007

Wow, that's the first time somebody called me "absolutely incompetent" with respect to Ice :eek:

grayclock · September 2007

The answers you are giving me are incorrect. RTFM doesn't solve the problem. In fact, I asked about the problem I am facing and you told me to read the FAQ. There is no solution of the possible bad port of ICE for c/c++ builder in the FAQ. I just asked for help, so can you help me? Other than that, ICE is amazing.

marc · September 2007

I already gave you the best advice I could give, to get and post the client log. If you think that this advice is incompetent, then I'm afraid there is not much I can do. I can't magically somehow find out what's going on without even seeing the log.

grayclock · September 2007

In addition, I compiled the following example, using C++ builder 2006:
$(iceroot)\demo\book\freeze_filesystem
It doesn't work! Stops in the middle with an exception!

grayclock · September 2007

If you don't wanna help me, I am not gonna bother you. I just don't know how to*explain my boss why I guaranteed that it was a good idea to do the project using ICE. Because of reasons, not depending on me, the client should be built using a C++ builder.

marc · September 2007

I told you (now for the third time), that in order to help you, I must see the client log. I don't know why you don't want to post it, but instead choose to attack me.

As for the Freeze problem, you should start a separate thread for this, and give the details of the error message you are seeing.

As an aside, I don't give a hoot about what you explain or don't explain to your boss...

grayclock · September 2007

I will prepare an example for you and send it. I cannot send you the whole source, because it is not mine. I would be really grateful if we could solve the problem.
Thank you in advance!
I apologize if I was offensive, I didn't mean to be

grayclock · September 2007

Marc, what do you mean saying client log? I mean... such a thing does not exist.

marc · September 2007

You posted the server log here. Get the same information for the client (Ice.Trace.Network=2 and Ice.Trace.Protocol=1) as explained in this FAQ.

matthew · September 2007

To diagnose connection failures you have know:
- What IP address & port the server is running on.
- What IP address & port the client is trying to connect with.

If these are not the same, then you know what the issue is and can work out how to solve this. If they are the same, and the connection attempt is failing there are various possibilities. A couple off the top of my head:
- A firewall is blocking connection attempts. This could either be locally, or remotely. You can help discover this by manually attempting to connect using telnet. For example, if your server is running on port 10000 on host 1.2.3.4 then you can run "telnet 1.2.3.4 10000".
- The connection takes some time and your timeout is configured too low.

Finally, there could be a compiler bug. As Marc previously told you the version of the Borland C++ compiler you are using is not supported and when we did the port to the prior version of this compiler we found it to be extremely buggy and had to discover numerous workarounds over the multitudes of problems that we discovered. Its entirely possible that there are either new bugs in the updated version of the compiler... The fact that another demo is failing for you does not bode well. Do the test suites run?

grayclock · September 2007

Matthew, the problem is not in the firewall, as I said in my previous posting. All firewalls were disabled and all the ports were open. If you look at the server log I posted, you will see that there is a connection between the client and the server. The problem is something else. I understand Marc's comment about CodeGear C++ builder 2007 too, but as I mentioned, there is the same problem with C++ Builder 2006, which is in the list of platforms you support.Yes, C++ Builder is a compiler full of bugs, so I will probably have to use VS C/C++. I will send you an example later, which you could use to reproduce the problem.
Thank you for your time.

marc · September 2007

I know that I probably sound like a broken record now, but get and post the client log, then we can narrow down the problem instead of guessing.

LAN/WAN problems

Comments