What is the best setup for minimizing the effect when a grid node is down

aozarov

When a grid node is down (crush, maintenance) what is the best strategy to minimize its effect on the clients (and save useless connection establishment tries). I should mention that we would like to distribute the load across the replicated nodes as much as possible and therefore would prefer not to cache the connection on the proxy and use the random selection strategy (as described in the manual section 37.3). I am aware that we can configure a frequent registry lookup to refresh the endpoints but wonder how quick the registry will be aware of the unavailability of the node and will it not include its endpoints. Also I wonder if there is a concept of "bad endpoint" which are not used temporarily in case of consecutive failures . Any other suggestions?
Thanks,
Arie.

benoit

How quickly the registry detects that a node is down depends on the IceGrid.Registry.NodeSessionTimeout property, see the Ice manual for more information on this property. Once the registry detects that a node is down, it won't return endpoints for servers deployed on this node.

Ice doesn't have any concepts of "bad endpoints". If you have a commercial interest in such a feature please contact us at [email protected].

To minimize connection attempts to invalid endpoints, you'll need to configure the client to lookup for server endpoints with the registry often and configure the registry node session timeout to a suitable value in order to detect shutdown nodes in a timely manner.

Cheers,
Benoit.

aozarov

Thanks!
I assume once the node is up again the registration (with the registry) is immediate (provided icegridnode is configured to run upon machine reboot)?

benoit

Yes, the node establishes a session with the registry on startup.

Cheers,
Benoit.