Archived

This forum has been archived. Please start a new discussion on GitHub.

IceGrid registry replica HA question

Hi,

We are considering to build RPC framework using ICE, and I'm currently looking for fail-safe IceGrid registry setup on amazon.

Basically, I want to build 1 master and 2 slaves (total 3 VMs) for the IceGrid registry.
There will be multiple IceGrid nodes (possibly multiple IceBox integrated), which will connect to the registry.

After briefly reading Ice manual, all IceGrid nodes and IceGrid registry should know their endpoints prior to being launched.

In the manual, IceGrid node configuration contains all endpoints of IceGrid registry (master and all slaves) before launching. If one of the registry slave fails (e.g. possibly VM crash or something), Question 1 how can I modify all IceGrid nodes to update this environment? (i.e. to remove the failed registry slave from the default locator)

One another thing is, after registry slave failed, we'll monitor the health of the registry nodes, and will create new VM for the new slave to the registry. This new registry slave will have different environment (e.g. different IP address). Question 2 How can I add this new registry slave endpoint information to the all IceGrid nodes?

Health checking of all VMs and notification for the IceGrid registry and IceGrid nodes will be handled by external software if ICE does not provide one. This should be no problem for our cases. However, I can't find a good tutorial or guide how to update the modified endpoints information for the IceGrid nodes/registry.

Regarding to the Question 2, my initial idea is, in a IceGrid node VM, if there is a notification of "registry endpoints updated", then the IceGrid node VM will query the list of the registry nodes using icegridadmin tool, and modify the node configuration and restart the IceGrid node. Question 3 Is there any way to update the IceGrid node configuration without restarting the grid node?

However, icegridadmin tool gives only the short hostname of the registry node, which is not enough to locate the registry (esp. in amazon). Question 4, is there anyway to get the FQDN of the registry?

Thanks,

Comments

  • benoit
    benoit Rennes, France
    Hi,
    cinsk wrote: »
    Hi,

    We are considering to build RPC framework using ICE, and I'm currently looking for fail-safe IceGrid registry setup on amazon.

    Basically, I want to build 1 master and 2 slaves (total 3 VMs) for the IceGrid registry.
    There will be multiple IceGrid nodes (possibly multiple IceBox integrated), which will connect to the registry.

    After briefly reading Ice manual, all IceGrid nodes and IceGrid registry should know their endpoints prior to being launched.

    In the manual, IceGrid node configuration contains all endpoints of IceGrid registry (master and all slaves) before launching. If one of the registry slave fails (e.g. possibly VM crash or something), Question 1 how can I modify all IceGrid nodes to update this environment? (i.e. to remove the failed registry slave from the default locator)

    It's not possible to update the list of registries used by an IceGrid node or IceGrid slave registry at runtime. You would need to restart them to update this list which is probably not what you want. If you use DNS names for the endpoints of each registry, you won't need to update anything other than the DNS database to eventually point to a new IP address. So with DNS names, you could for example use the following for the default locator property:
         IceGrid/Locator:tcp -h master.domain.com:IceGrid/Locator:tcp -h slave1.domain.com:IceGrid/Locator:tcp -h slave2.domain.com
    

    One another thing is, after registry slave failed, we'll monitor the health of the registry nodes, and will create new VM for the new slave to the registry. This new registry slave will have different environment (e.g. different IP address). Question 2 How can I add this new registry slave endpoint information to the all IceGrid nodes?

    Health checking of all VMs and notification for the IceGrid registry and IceGrid nodes will be handled by external software if ICE does not provide one. This should be no problem for our cases. However, I can't find a good tutorial or guide how to update the modified endpoints information for the IceGrid nodes/registry.

    Regarding to the Question 2, my initial idea is, in a IceGrid node VM, if there is a notification of "registry endpoints updated", then the IceGrid node VM will query the list of the registry nodes using icegridadmin tool, and modify the node configuration and restart the IceGrid node. Question 3 Is there any way to update the IceGrid node configuration without restarting the grid node?

    No, there's no way to do this at this time. As suggested above, the best to deal with this is to rely on DNS names and update the IP associated with the DNS name of the registry VM that was restarted. This way you don't have to change the configuration of the IceGrid node and slaves.

    The update of the DNS database can be done dynamically when the new VM is started. Your VM startup scripts could register the VM with your local DNS server which keeps track of VMs started for your application in the EC2 cloud.
    However, icegridadmin tool gives only the short hostname of the registry node, which is not enough to locate the registry (esp. in amazon). Question 4, is there anyway to get the FQDN of the registry?

    Thanks,

    It's not possibly to return the FQDN at this point, it only returns the node name as returned by the "uname -n" command.

    Let us know if you need more information on this!

    Cheers,
    Benoit.