Archived
Set QueueSizeMax per topic in IceStorm
Hello,
in our application we a lot of different topics with different frequencies and amounts of data. Also some clients process the data in different frequencies.
1) Is there a way to set something similar to service.Send.QueueSizeMax per topic from the source code?
2) Is there a way to set this per receiving client on per topic?
Looking at TopicManager::create(string) and Topic in the slice interface and did not see an option. Did i miss something?
Thanks for all hints
Comments
-
Hi,
You didn't miss something, there's currently no way to set this at the topic or subscriber level, it's a global IceStorm configuration. The queue size max configuration was designed as a protection for IceStorm to avoid consuming too much memory if a client doesn't consume its events or not fast enough.
Can't you set this property to a value large enough to satisfy all the subscribers? Or do you want to rely on this property to unsubscribe or drop events events for slow subscribers automatically?
As a workaround, you could instantiate different IceStorm service instances in the same IceBox server with different configurations but I understand that it's not ideal.
Note that we also released recently a new service called DataStorm, you might want to take a look at it as well.
Cheers,
Benoit.5 -
Hi,
thanks for the info. I noticed my question could have been mare clear (describing what i want to achieve).
I want to drop events for slow subscribers.Basically we have two kinds of topics:
1) We want all data to arrive at the subscriber (means no dropping except if too much data piles up)
2) We only need the newest data (basically set the queue to 1)For some debugging / logging components it would be nice to get all data for topics of type 2. That is why i asked about per subscriber configuration.
I will have a look at DataStorm and maybe we will look at instantiating two IceStorm servers.
0
