Who will reap the abandoned resource?

OrNot

hi, there,
Michi once wrote an instructive article in one of Connections to discuss the reaper issue. It is a really good article.
I have similar question about tcp sockets. If a client crashs without sending any colse signal to server, what's the fate of the sokcet once belonging to that client? It will be automatically reaped by OS kernel later ? how soon ? Can this be controled by us?

Thank you .

OrNot

marc

If the client crashes, and the operating system detects the crash, the client's operating system will send a TCP FIN to the server, which causes the server to detect the disruptive connection closure, and clean up all resources associated with this connection.

If the whole client computer crashes, or the Internet connection to the client goes down, then the server will never receive a TCP FIN. There is the TCP Keep Alive timeout which would eventually cause the server to close the socket, but this timeout is very large (like several hours), so you can't rely on this.

You can either use active connection management on the server side, which will make sure that idle connections are closed after a certain timeout, or you can close connections explicitly in your application's code when your server detects that a client is dead and reaps all associated objects.

OrNot

hi, Marc,
Here is our scenario. Could you please give me some advice?
hundreds of clients are connected to a server with bidir mode and every client is forced to register a callback periodically on the server. As the manu documents, the ACM is not recommended in bidir mode and I disabled it. Then it is my duty to reap the abondoned sockets. But it is not very easy to manage the connections since I have to maintain them.
I read your post and realize that my understanding seems wrong. Since I have foreced the clients to connect sever periodically, I can certainly enable the Ice.Server.ACM and let it do the socket reap work for me and what I need is only to reap other resources such as callbacks . everything will be ok only if the Ice.Server.ACM timeout is set same as my reap period.
I am not very sure about this and hope getting your confirmations.
Thanks a lot.

OrNot

OrNot

sorry, another related quesiton.

Add the following to include/IceUtil/Config.h, before Windows.h is included:

#define FD_SETSIZE 1024

Then you can have up to 1024 simultaneous connections.

If you need even more than that, then it might be advisable to activate connection management, so that idle connections are closed and re-opened on demand.

In our scenario, maybe more then 1024 simultaneous connections. And is it right to set it to, say, 5000? Any dangers?

marc

If you use bi-dir connections, and already have a reaper like the one from the Connections article, you can simply close the connection along with all the per-client objects you destroy. Just use one of the callback proxies to get the bi-dir connection to the client, and explicitly call close for this connection. Use timeouts, so that the call to close doesn't hang if the client doesn't react.

As for the number of connections, if your server can handle a load of 5,000 simultaneous clients, I don't see any problems. It really depends on what the server is doing for the clients. If the clients send a lot of computing-intensive requests, then 5,000 may be way too many. If the clients are mostly idle and mostly execute simple requests, then your server can probably handle more than 5,000 clients.

You could also use Glacier, and let Glacier worry about connection management and bi-directional connections. You could then also use several Glacier instances as connection concentrators, so that your server only sees a few connections, and some other dedicated front-end computers handle all the traffic with the clients. Again, this might be an overkill, depending on your application.

OrNot

If you use bi-dir connections, and already have a reaper like the one from the Connections article, you can simply close the connection along with all the per-client objects you destroy. Just use one of the callback proxies to get the bi-dir connection to the client, and explicitly call close for this connection. Use timeouts, so that the call to close doesn't hang if the client doesn't react.

Oh, Yeah! Very Nice! Very Nice!

As for the number of connections, if your server can handle a load of 5,000 simultaneous clients, I don't see any problems. It really depends on what the server is doing for the clients. If the clients send a lot of computing-intensive requests, then 5,000 may be way too many. If the clients are mostly idle and mostly execute simple requests, then your server can probably handle more than 5,000 clients.

My server is actually a message router just like glacier2 so I think it is a good news for it to handle so many clients. By the way, FD_SETSIZE can be safely set more than 1024 in windows as you posted, but it should be recompile with OS kernel in linux, shoudn't it?

marc

I don't know what the Linux limit is. It might depend on your distribution and kernel version. I believe the default is much higher than 1024. Please check your Linux kernel documentation for details.

bernard

OrNot wrote:

Add the following to include/IceUtil/Config.h, before Windows.h is included:

#define FD_SETSIZE 1024

Then you can have up to 1024 simultaneous connections.

If you need even more than that, then it might be advisable to activate connection management, so that idle connections are closed and re-opened on demand.

Please note that on Windows Ice is built with FD_SETSIZE set to 1024 unless you update the build system to change this value. And the proper way to change FD_SETSIZE is not to define it in IceUtil/Config.h; define instead the desired value when building Ice and IceSSL (see the Ice and IceSSL project files).

Cheers,
Bernard

OrNot

Thank you Marc, Thank you Bernard.
hi, Bernard,
do you mean Ice has silently modified FD_SETSIZE to 1024 in windows for us?


Another question.
What will happen if FD_SETSIZE is 64 but we allow more then 64 clients connect to server (without reaper)? the 65th client will be refuesed to connect or just blocked there to wait one of 64 die?