IceStorm - durable subscription

Hi,

Note that the subscriber is automatically un-subscribed from IceStorm if IceStorm gets an Ice::ObjectNotExistException or Ice::NotRegisteredException exception when invoking on the subscriber (these are considered hard failures). For other exceptions, IceStorm uses the retryCount QoS to figure out when the subscriber should be unsubscribed. In any case, while a subscriber is un-reachable the events are discarded and the delivery resumes only once the subscriber is available again. For more information on this retryCount QoS, see here in the manual.

Perhaps you can detail a little why you need event queuing while the subscriber is down? An alternative solution to this might be to use per-subscriber publishers. Basically, the idea is that each time a subscriber comes online, it contacts the publisher to retrieve the latest state of the publisher. The publisher sends this information through the per-subscriber publisher object. Once it has retrieved the latest state, the subscriber can start receiving and processing again updates. This works well when using IceStorm to implement the observer pattern and avoids having to queue updates in the IceStorm service. The manual describes an example here. The C++ demo/IceStorm/counter demo also demonstrates how to use it.

Cheers,
Benoit.

The important thing to remember here is that there is no free lunch. In my experience in implementing exactly such as system (a notification system with durable events) there is a very big cost which you pay for such durability in terms of raw system throughput. In particular, if you want guarantees that the messages won't be lost, or even worse the messages respect transaction boundaries (in the case of distributed transactions) throughput can drop from thousands of events per second, to much much lower.

In most cases there is a better way to design your system (as Benoit pointed out) where you don't have to pay such a high cost.

Another thing to note is that IceStorm is specifically not a message queue. Its a publish/subscribe system. You should make sure you are comparing apples to apples.