Using AMI and normal invocations on the same proxy from different threads

andreynechandreynech Member Andrey NechypurenkoOrganization: Veterobot.comProject: Robotics vehicle for researchers and makers ✭✭
Hi,

In my application I am using two threads collecting data from different sensors. The reason to have two threads is different scan periods, i.e. 20Hz and 30Hz. Both threads are using the same callback proxy object to send collected data out.

For some reasons, I decide to use AMI in one thread and synchronous calls in other thread. In some cases it looks like synchronous call gets blocked forever. That is why, before getting deeper in debugging, I would like to ask a couple of questions.

The first is whether my assumption that Ice (in my case it is IceE 1.3.0 on Linux) proxy interfaces are thread safe is correct? The second is whether mixing AMI and synchronous calls from different thread is something bad or it is a provided scenario (if necessary, I can provide more details why I decide to do it this way)?

I would really appreciate any hints about possible reasons for the blocking behavior and maybe suggestions how to fix it. In the past, I've seen similar problem with callbacks which was fixed by increasing the size of thread pools on the client and server. In this case, it does not help.

Thank you,

Comments

  • benoitbenoit Rennes, FranceAdministrators, ZeroC Staff Benoit FoucherOrganization: ZeroC, Inc.Project: Ice ZeroC Staff
    Hi Andrey,

    Yes proxies are thread-safe and mixing AMI and regular synchronous calls should be fine.

    You'd need to attach to your process with a debugger and see the stack traces of each thread to figure out what is wrong there.

    That being said, one likely reason for this hang is a threading problem: Ice client thread pool thread exhaustion because of a deadlock. AMI callbacks are dispatched by the Ice client thread pool (whose size is by default 1). If the AMI callback tries to acquire a lock and this lock is currently held by another thread, the client thread pool thread will hang until it can acquire the lock. Meanwhile, no more responses from servers can be received and regular synchronous calls will also hang. If the thread holding the lock is the thread making the synchronous call, you end up with a deadlock.

    To fix this, you have 2 solutions:
    • You increase the client thread pool size to a size big enough to ensure you can't run into this problem.
    • You fix your program to not hold locks (that need to be acquired by AMI callbacks) while synchronous Ice calls.

    Cheers,
    Benoit.
  • andreynechandreynech Member Andrey NechypurenkoOrganization: Veterobot.comProject: Robotics vehicle for researchers and makers ✭✭
    Hi Benoit,

    Thank you very much for the information! It looks like I can not blame Ice for the hang up and need to get deeper in debugging :)

    Thank you,
    Andrey.
Sign In or Register to comment.