Archived

This forum has been archived. Please start a new discussion on GitHub.

unsubscribing from IceStorm

Hi,

I'm having trouble trying to reliably unsubscribe from IceStorm.
I'm using IceStorm in one-way mode, for Ice v3.1.1.

I have a class that subscribes an Object, and I'd like it to clean up after itself in its destructor. As far as I can tell there are two things it can do:

1) Call IceStorm's 'unsubscribe()', and
2) remove the Object from the adapter.

The first option is a little awkward, because it involves a remote call. If something temporarily goes wrong (eg with the network), it's difficult to recover. I could potentially have a separate thread which loops forever, trying to unsubscribe about-to-be-destructed classes, but that's a bit messy.

The second option doesn't work for me because I'm using IceStorm in one-way mode. So presumably IceStorm's call would raise an ObjectNotExist exception, but IceStorm never sees it (and therefore never unsubscribes the non-existent Object).

The result of neither option working is that my network usage slowly rises over the lifetime of my application. As I subscribe and (try to) unsubscribe Objects, the number of successful subscriptions becomes greater than the number of successful unsubscriptions. My application therefore receives a lot of useless data from IceStorm which is never read by any of my code.


I realise that a third option is to use two-way semantics, but I'd prefer to stick with one-way. Is there a fourth option?


Thanks very much for any help,

Alex

Comments

  • xdm
    xdm La Coruña, Spain
    Hi Alex

    from Ice-3.1.1.pdf
    42.3.7 Subscriber Errors
    If IceStorm encounters a failure while attempting to deliver a message to a
    subscriber, the subscriber is immediately unsubscribed from the topic on which
    the message was published.

    Slow or unresponsive subscribers can also create problems. IceStorm attempts
    to deliver messages as soon as they are published: when the IceStorm publisher
    object receives a message, it may immediately make nested invocations to the
    topic’s subscribers. Therefore, if a subscriber is not consuming messages as fast as
    they are being published, it may cause threads to accumulate in IceStorm. If the
    number of threads reaches the maximum size of the thread pool (see
    Section 30.9), then no new messages can be published. You may also want to set a
    timeout in IceStorm’s configuration (using Ice.Override.Timeout) so that
    an unresponsive subscriber does not block the service indefinitely.

    if documentation is correct IceStorm automatic remove unreachable Subscribers. I think your problem must be in other place. But is dificult to know with out more details

    Hope this helps
  • Thanks for the suggestion. This is true, IceStorm automatically removes unreachable subscribers. But in my case the subscriber is reachable: since my application is still running, the adapter is still alive and well, and can be reached by IceStorm. A problem only occurs when the adapter realises that ther's no Object to which to give the message. It raises an ObjectNotExist exception, but this never reaches IceStorm when one-way semantics are used.
  • benoit
    benoit Rennes, France
    Hi,

    If you can't unreliably unsubscribe from IceStorm, the only option is to use twoway for the delivery mode. Twoway adds only a little overhead (IceStorm uses AMI to deliever the updates with twoway calls).

    Cheers,
    Benoit.
  • matthew
    matthew NL, Canada
    The upcoming version of IceStorm features some performance enhancements and in high throughput situations one-way events will be much faster than twoway. For the Ice 3.1.1 version of IceStorm this is, however, not the case as Benoit says.

    I don't understand your scenario. If the server is not running IceStorm will detect this and kick the subscribers. I guess the problem you are running into is that the server crashes and is restarted before IceStorm publishes further events. This indicates to me that you are subscribing incorrectly. Perhaps you are using indirect proxies, or you are re-using the same port all the time for the subscriber OA. In both of these cases don't. Subscribers are not persistent objects.
  • Hi Matthew,

    Sorry, maybe I can try to make my scenario really clear.

    In all cases I do the following:
    1) Start IceStorm
    2) Start MyPublisher, which runs forever
    3) Start MySubscriber, which runs forever

    MySubscriber does something like this (paraphrased with pseudo-code)
      bool unsub  = true;
      bool remove = true;
      int numTimes = 1;
    
      for ( int i=0; i < numTimes; i++ )
      {
        Ice::ObjectPtr consumer = new MyConsumerI;
        Ice::ObjectPrx obj = adapter->addWithUUID( consumer );
    
        Ice::ObjectPrx prx = adapter->createDirectProxy( obj->ice_getIdentity() );
        MyConsumerPrx callbackPrx = MyConsumerPrx::uncheckedCast( prx );
    
        subscribeToIceStorm( callbackPrx );
        if ( unsub )
          unsubscribeFromIceStorm( callbackPrx );
        if ( remove )
          adapter->remove( obj->ice_getIdentity )
      }
    
      ... wait for data to roll in ...
    

    Now I monitor the amount of traffic running through my computer's interface. Since I'm running everything on the same host, I'm just looking at the loopback interface.

    As written above, I see n kbps, where n is the traffic produced by MyPublisher (and sent to IceStorm).

    If I set:
    - unsub = false
    - remove = false
    then I see 2n kbps: traffic to IceStorm, plus traffic from IceStorm to MySubscriber.

    If I set:
    - unsub = false
    - remove = true
    then I still see 2n kbps. In other words, IceStorm doesn't realise that my Object has been removed from the adapter, and still happily delivers the data.

    If I set:
    - unsub = false
    - remove = true
    - numTimes = 10
    then I see 11n kbps. In other words, IceStorm delivers the data 10 times, even though no-one is listening. Since the adapter is still alive, it doesn't kick the subscriber.



    Does this make more sense?


    Thanks,

    Alex
  • matthew
    matthew NL, Canada
    Yes, but the key question is why does your program not reliably unsubscribe in the event that it does not crash? Really the only case where this should occur is a temporary network interruption. In this case you could remember that the unsubscribe failed and retry at a later point.
  • I can't reliably unsubscribe due to network issues: I'm running a set of mobile robots using a wireless network, so sometimes there are interruptions.
    I think your suggestion is as I mentioned in my first post: have a thread which loops forever trying to unsubscribe an arbitrary set of about-to-be-destructed objects. This is pretty awkward though, I was hoping to avoid this. Is there no other way?

    Thanks again for the help,

    Alex
  • matthew
    matthew NL, Canada
    At present there is no other way, except to use two way as Benoit suggested.
  • OK. Thanks very much for the help clearing things up Matthew & Benoit.


    Alex