Archived

This forum has been archived. Please start a new discussion on GitHub.

Ice-python looses threadlocals

Hi,
it seems that threadlocals (instances of python's threading.local()) are not preserved in between calls from Ice runtime to python runtime for the same thread.

Simple example based on printer demo:
import sys, traceback, Ice
import threading
from pprint import pprint, pformat

#Ice.loadSlice('Printer.ice')
#import Demo

#class PrinterI(Demo.Printer):
#    def printString(self, s, current=None):
#        print s

class ThreadLogger(Ice.ThreadNotification):

    def __init__(self):
        super(ThreadLogger, self).__init__()
        self._tls = threading.local()

    def start(self):
        print "ice thread started: " + repr(threading.current_thread()) + " thread locals:"
        self._tls.my_test_var = "This won't survive until stop()"
        pprint(self._tls.__dict__)

    def stop(self):
        print "ice thread stopped: " + repr(threading.current_thread()) + " thread locals:"
        pprint(self._tls.__dict__)
    

    

status = 0
ice = None
try:
    initData = Ice.InitializationData()
    initData.threadHook = ThreadLogger()
    ic = Ice.initialize(sys.argv, initData)
    adapter = ic.createObjectAdapterWithEndpoints("SimplePrinterAdapter", "default -p 10000")
    #object = PrinterI()
    #adapter.add(object, ic.stringToIdentity("SimplePrinter"))
    adapter.activate()
    ic.waitForShutdown()
except:
    traceback.print_exc()
    status = 1

if ic:
    # Clean up
    try:
        ic.destroy()
    except:
        traceback.print_exec()
        status = 1

sys.exit(status)

To reproduce the bug please run the above script - it should print that it has set up a threadlocal var for two ice threads. Next, please interrupt the script, which will cause ice runtime to call the threadHook.stop() and method shut down the threads that it has started. The threadlocal value set up in threadHook.start() is no longer present when calling threadHook.stop().

Using threadHook is the simplest and shortest way to demonstrate this behaviour, but is also occurs for example in servant locators - in this case threadlocals are not preserved between calls to locate() and finished().

Tested on win xp 32bit:
- Ice 3.4.1 official binary downloaded from the site
- Python 2.6.6 32-bit official binary

and mac os x 10.6.7:
- Ice 3.4.1 from MacPorts
- Python 2.6.6 from MacPorts

Comments

  • mes
    mes California
    Filip,

    Thanks for reporting this.

    Cheers,
    Mark
  • It seems, that the problem occurs only when python is using thread._local as the implementation of threading.local. I guess thread._local is more optimized and os-specyfic. However, if I force python to use the default (and probably much slower) implementation, threadlocals are not lost:
    import threading
    import _threading_local
    threading.local = _threading_local.local
    
    
    import sys, traceback, Ice
    from pprint import pprint, pformat
    
    class ThreadLogger(Ice.ThreadNotification):
        def __init__(self):
            super(ThreadLogger, self).__init__()
            self._tls = threading.local()
    
        def start(self):
            self._tls.my_test_var = "_threading_local.local works"
            print "ice thread started: " + repr(threading.current_thread()) + " thread locals: " + pformat(self._tls.__dict__)
    
        def stop(self):
            print "ice thread stopped: " + repr(threading.current_thread()) + " thread locals: " + pformat(self._tls.__dict__)
        
        
    status = 0
    ice = None
    try:
        initData = Ice.InitializationData()
        initData.threadHook = ThreadLogger()
        ic = Ice.initialize(sys.argv, initData)
        adapter = ic.createObjectAdapterWithEndpoints("SimplePrinterAdapter", "default -p 10000")
        adapter.activate()
        ic.waitForShutdown()
    except:
        traceback.print_exc()
        status = 1
    
    if ic:
        # Clean up
        try:
            ic.destroy()
        except:
            traceback.print_exec()
            status = 1
    
    sys.exit(status)
    
    

    I wonder if this bug is strictly ice-specific, or is python also involved?
  • Hi, we are experiencing the same problem using Ubuntu 14.04 (64 bit) and its official zeroc-ice35 and python-zeroc-ice packages (version 3.5.1-5.2).

    The problem can be easily reproduced by putting the following into any method called from Ice:
    import threading
    x = threading.local()
    if not hasattr(x, 'counter'): x.counter=0
    x.counter += 1
    print 'local: %s' % x.__dict__
    

    One would expect to see the counter increasing with every method call.
    In reality, it is initialized freshly every time with 1.

    This occurs even if the default single-thread model is used which can be confirmed with
    --Ice.Trace.ThreadPool=1:
    "ThreadPool: creating Ice.ThreadPool.Server: Size = 1, SizeMax = 1, SizeWarn = 0 ]".
    Also, thread.get_ident() returns the same thread ID for every method invocation.

    Unfortunately, the workaround of filip84 (using _threading_local) also does not work for us.
    The problem is a show-stopper for our software, since it breaks the database connection handling inside Django.

    Does anybody have a working workaround for this problem, or maybe even a fix?