Archived

This forum has been archived. Please start a new discussion on GitHub.

IceStorm’s replication with IceGrid's Registry Replication

When I use IceStorm’s replication with IceGrid's Registry, there must be more than 3 IceStorm instances, or IceStorm will give some warnings and exit.
And, when I configure the IceStorm’s replication 3 instances, one node for 1 instance and the other for 2, only start node 1, then the IceStorm try connecting to node2 time after time and the demo publisher can't run correctly, and stop on invoking "topic = manager->retrieve(topicName)".
So my 2 questions:
1. Can only 2 IceStorm instances run together?
2. When only some of the configured instances(eg, 1 of 3) run, why the publisher can't invoke the "manager->retrieve(topicName)", but is connecting?

Comments

  • matthew
    matthew NL, Canada
    Both of these are expected, and are an artifact of the implementation.

    Replication requires a minimum of three nodes. If only one replica is running, then you may not make any state changes on the IceStorm database. I explain why this occurs in my article on HA IceStorm in Issue 28 of Connections http://www.zeroc.com/newsletter/issue28.pdf

    If you still have questions after reading this please let me know!
  • thank you for the reply.
    Article "Highly Available IceStorm" has talked about this.

    But for a small application, there are only 2 servers(2 nodes), so to use IceStorm for processing events, then I can make 2 IceStorm instances in one node, is this rational? maybe some other solutions?
  • If make 2 IceStorm instances in node1, 1 instance in node2, then once some errors occur on node1, then the IceStorm would stop service too.

    cheers.
  • matthew
    matthew NL, Canada
    Certainly for small deployments you should be able to put multiple IceStorm instances on the same host. This should work, and if it does not there is some problem that should be investigated. Note, that this is actually tested by the test suite, and demonstrated by the demos so it should work.

    Note that such a deployment, however, does not protect you very well from hardware failures.
  • Election Algorithm Modification

    I'm not 100% sure if this is the thread to pose this question, but this thread seems to be related to my issue. I am currently working on a training system for multiple combat systems. One of the requirements is that individual combat systems can train standalone or with the other combat systems. Additionally, individual systems can enter and exit a running training scenario. Due to this, there is no way to ensure that all ice grid nodes will be on at system startup. There is also no way to require that a given set of computers will be on at any point in the training session.

    However, if each combat system had its own ice grid running replicated ice storms, then the individual systems could replicate amongst themselves. Then, when systems are added to the training scenario, the individual replica groups could link up. When a system exits the training scenario, its system-specific replica group could then be removed from the overall replica group. In order for this to work, the replica groups on the individual system would have to have the capability to join and leave other replica groups on the fly.

    Are there any plans for this sort of election algorithm being implemented in future versions of Ice? If not, is there a technical reason (not including cost / lack of desire for this feature by the community) that would cause this sort of replication to be a bad idea? Is there an easier way to achieve this (possibly by writing a process that pushes information between replica groups)?

    I know that the example I've given may not be applicable to most developers, but this same issue is related to the idea of having "cold" spares which may need to replicate data, but aren't guaranteed to be on at system startup.
  • matthew
    matthew NL, Canada
    That is quite a different algorithm than the algorithm used by IceStorm. IceStorm requires all replicas to be up and active to guarantee that the latest database state is used (if this was not the case database divergence could occur, leading to very bad results).

    At this point, we have no plans to change the replication algorithm used by IceStorm. However, if you have a commercial need for such a change you can contact sales@zeroc.com to discuss this project.
  • Thanks for your quick response. I don't have a commercial need for this. I've been playing around with this, and I think I can do something similar without using a new election algorithm, but I need to be able to change the IceGrid.Registry.ReplicaName property on a running replica group. It doesn't appear that this is possible with the current implementation.
  • bernard
    bernard Jupiter, FL
    Hi Harry,

    I think your requirement is actually unrelated to replication. Replication in Ice (including IceGrid and IceStorm) is about providing fail-over and load-balancing: for example, if your application uses an IceStorm replica and this replica crashes or becomes unavailable for some reason, your application can continue with another (identical) replica.

    When you connect several training systems, each with its own IceGrid and IceStorm services, there is no potential for "cross replication"--the IceGrid and IceStorm services in these separate systems can't be identical, so can't become replica of each other.

    There are however interesting issues when connecting these systems, for example:
    - how do events from one system flow to the other (possible answer: federate the IceStorm instances using links)
    - how do clients in one system resolve indirect proxies that refer to objects in another system (try to avoid the problem altogether, or perhaps use several Ice Locator proxies in your clients)

    Best regards,
    Bernard