Problems compiling Ice 3.3 from source on Debian

yamokosk

Greetings,

I recently tried upgrading my Ice installation from 3.2 to 3.3. Compiling goes fine until this:

c++ -Wl,--enable-new-dtags -Wl,-rpath,/opt/Ice-3.3/lib -ftemplate-depth-128 -Wall -D_REENTRANT -I/usr/include/nptl -fPIC -g -L../../lib -o ../../bin/slice2cpp Gen.o Main.o -lSlice -lIceUtil -L/usr/lib/nptl -lpthread -lrt
/lib/librt.so.1: undefined reference to `__librt_enable_asynccancel@GLIBC_PRIVATE'
/lib/librt.so.1: undefined reference to `__librt_multiple_threads@GLIBC_PRIVATE'
/lib/librt.so.1: undefined reference to `__librt_disable_asynccancel@GLIBC_PRIVATE'

Here are some details of my setup:

yamokosk@borelli:~/src/Ice-3.3.0/cpp$ g++ --version
g++ (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)

yamokosk@borelli:~/src/Ice-3.3.0/cpp$ cat /proc/version
Linux version 2.6.23.17-oblab

Not really sure why my version of librt.so (/lib/librt-2.3.6.so) is not what Ice is looking for. On Debian, librt is part of the libc6 package... pretty much a core package for the system. What version of librt does Ice 3.3 need?

bernard

Hi JD,

Ice 3.3 doesn't require a specific version of librt. It does however require a recent version of the Native POSIX Thread Linux (because we added monotonic clock support in Ice 3.3).

On Red Hat Enterprise Linux 4 (and CentOS 4), we need to include the pthread.h in /usr/include/nptl and link with the libpthread in /usr/lib/nptl to get the required functions.

Unfortunately our build system uses this extra include directory and library directory on all Linux distributions. Usually it's not an issue (these directories don't exist), but it's possible that on your system you have a version of the Pthread library in these directories that is incompatible with your librt.

So start by checking if /usr/include/nptl or /usr/lib/nptl exists on your system; if that's the case, edit cpp/config/Make.rules.Linux, remove all nptl settings, "make clean" and try again.

Cheers,
Bernard

yamokosk

More problems

Well you were right. I do have a /usr/include/nptl directory on my machine. But I'm afraid I have run into more problems now. This time the compiler is complaining:

c++ -c -I../../include -DICE_UTIL_API_EXPORTS -I.. -ftemplate-depth-128 -Wall -D_REENTRANT -fPIC -g Cond.cpp
Cond.cpp: In constructor ‘IceUtil::Cond::Cond()’:
Cond.cpp:311: error: ‘pthread_condattr_setclock’ was not declared in this scope

I changed several lines in the make file you recommended from:

CXXFLAGS = $(CXXARCHFLAGS) -ftemplate-depth-128 -Wall -D_REENTRANT -I/usr/include/nptl
...
BASELIBS = -lIceUtil -L$(NPTL_LIB) -lpthread -lrt

to:

CXXFLAGS = $(CXXARCHFLAGS) -ftemplate-depth-128 -Wall -D_REENTRANT
...
BASELIBS = -lIceUtil -lpthread -lrt

But I am guessing that I need to not just remove the include of nptl but also replace it with something else..

bernard

Hi JD,

It looks like the default Pthread header/library on your Debian system don't include the required functions, like on RHEL4.

So I recommend to install first /usr/lib/nptl/librt.so.1, and then try again to build with /usr/include/nptl and /usr/lib/nptl.

Cheers,
Bernard

bernard

Actually, since you want to link with the NPTL/TLS librt.so.1, you need a librt.so symbolic link to this library. Unfortunately I don't where to find it on a Debian system.

Cheers,
Bernard

yamokosk

Confused..

I am sorry if I am being really dense about this but isn't that how I got into this linking problem in the first place? On my system, the nptl directories already exist. And for whatever reason, when Ice links against librt it fails to find those symbols it needs.. unless there are more than on version of librt floating around on my system?? A nptl version and non-nptl version???

The reason those nptl directories exist is because I already have libc6-dev installed. libc6-dev installs, among other things:

/usr/lib/libpthread.so
/usr/lib/nptl
/usr/lib/nptl/libc.a
/usr/lib/nptl/libc_nonshared.a
/usr/lib/nptl/libpthread.a
/usr/lib/nptl/libpthread_nonshared.a
/usr/lib/nptl/librt.a
/usr/lib/nptl/libc.so
/usr/lib/nptl/libpthread.so
...
/usr/include/nptl
/usr/include/nptl/bits
/usr/include/nptl/bits/libc-lock.h
/usr/include/nptl/bits/stdio-lock.h
/usr/include/nptl/bits/typesizes.h
/usr/include/nptl/bits/pthreadtypes.h
/usr/include/nptl/bits/posix_opt.h
/usr/include/nptl/bits/local_lim.h
/usr/include/nptl/bits/initspin.h
/usr/include/nptl/bits/semaphore.h
/usr/include/nptl/pthread.h
/usr/include/nptl/semaphore.h
/usr/include/nptl/thread_db.h
...
/usr/lib/librt.so


Please guide me out of the darkness!

yamokosk

Ahh.. just saw your second message. So I do need to point to the nptl version of librt. It looks like there is just a static library created at:

/usr/lib/nptl/librt.a

Is this what I need to link against?

bernard

Hi JD,

From your first message, it does not look like you're linking with /usr/lib/nptl/librt.a ... not sure why, however you're probably better off with the (proper) shared library.

You really want to link with /lib/tls/librt.so.1, not /lib/librt.so.1. However, I don't know what's the proper way to do this on your Debian system. If you have LD_ASSUME_KERNEL set, unset it; see Explaining LD_ASSUME_KERNEL

Cheers,
Bernard

yamokosk

Fixed!

I fixed it! Well at least to the point where it is now compiling. I messed around with the Make.rules.Linux file and forced everywhere there was a "-lpthread -lrt" to search either /lib/tls or /usr/lib/nptl. Here are the results:

/lib/tls: ERROR!

c++ -Wl,--enable-new-dtags -Wl,-rpath,/opt/Ice-3.3/lib -ftemplate-depth-128 -Wall -D_REENTRANT -I/usr/include/nptl -fPIC -g -L../../lib -o ../../bin/slice2cpp Gen.o Main.o -lSlice -lIceUtil -L/lib/tls -lpthread -lrt
../../lib/libIceUtil.so: undefined reference to `pthread_condattr_setclock'

So libIceUtil.so gets built properly. And in fact if I ldd libIceUtil.so I see:

yamokosk@borelli:~/src/Ice-3.3.0/cpp/lib$ ldd libIceUtil.so
linux-gate.so.1 => (0xffffe000)
libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7e8b000)
librt.so.1 => /lib/tls/i686/cmov/librt.so.1 (0xb7e82000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7d9d000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7d78000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7d6d000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7c3c000)
/lib/ld-linux.so.2 (0x80000000)


/usr/lib/nptl: WORKS!

I think what was going wrong before is that some linking steps used:

-lpthread -lrt

while others used:

-L$(NPTL_LIB) -lpthread -lrt

And on my system those were two different sets of libraries. I haven't yet run the new binaries.. hopefully I won't get any runtime errors...

volk

I've had exactly the same problem on Debian. I have resolved it by creating a file named librt.so (without any prefixes/suffixes) in /usr/lib/nptl:

/* GNU ld script
   Use the shared library, but some functions are only in
   the static library, so try that secondarily.  */
OUTPUT_FORMAT(elf32-i386)
GROUP ( /lib/tls/librt-2.3.6.so /usr/lib/nptl/librt.a )

(/usr/lib/nptl already has files libc.so and libpthread.so with approximately the same contents).

yamokosk

Ice 3.3.1?

So is this the official stance on this problem? I have no problem doing something like this but just want to make sure there is not a better way.

jonaf1

Hello all,

I'm experiencing this problem as well. I'm on Debian Etch with gcc 4.1.2.

I first received the "undefined reference to `__librt_enable_asynccancel@GLIBC_PRIVATE'" and related errors mentioned in the first post of this thread.

I then created the script mentioned by volk in 2 posts up. I was then able to get Ice to compile successfully.

So my current rt lib structure:

$ ls -l /usr/lib/nptl/librt*
-rw-r--r-- 1 root root 58388 2008-01-19 06:14 /usr/lib/nptl/librt.a
-rw-r--r-- 1 root root   222 2008-06-26 03:24 /usr/lib/nptl/librt.so
lrwxrwxrwx 1 root root    19 2008-03-01 11:54 /usr/lib/nptl/librt.so.1 -> /lib/tls/librt.so.1

$ ls -l /lib/tls/librt*
-rw-r--r-- 1 root root 26516 2008-01-19 06:14 /lib/tls/librt-2.3.6.so
lrwxrwxrwx 1 root root    14 2008-03-01 11:54 /lib/tls/librt.so.1 -> librt-2.3.6.so

The problem is when I try to use Ice (using cmake which checks if it can include Ice/Ice.h), I get linker errors / undefined references to pthread_condattr_setclock from libIceUtil:

/usr/bin/c++     -I/opt/Ice-3.3/include -L/opt/Ice-3.3/lib -L/opt/Ice-3.3/lib64 -lIce -lIceUtil    -fPIC CMakeFiles/cmTryCompileExec.dir/CheckIncludeFile.cxx.o  -o cmTryCompileExec -rdynamic
/opt/Ice-3.3/lib/libIceUtil.so: undefined reference to `pthread_condattr_setclock'
collect2: ld returned 1 exit status

I'm not sure what to do. Should I just get rid of rt/nptl/tls and try to install all of these from source, as it seems to be a debian issue?

yamokosk, can you perhaps post your Make.rules.Linux file? I am currently compiling by adding -L$(NPTL_LIB) before any occurrence of -lpthread -lrt in Make.rules.Linux, we'll see how it goes.

jonaf1

The above compiled but I still got the same linker error, but received extra info after the function name:
/opt/Ice-3.3/lib/libIceUtil.so: undefined reference to `pthread_condattr_setclock@GLIBC_2.3.3'

Edit: After more trial & error, I finally got cmake to correctly test for the inclusion of Ice/Ice.h. The thing is I have to specify the nptl library path (/usr/lib/nptl), and the libs rt and pthread anytime I use IceUtil. Is this how it should be (e.g., explicitly specifying these options all the time)?

bernard

Hi Jonaf,

Yes, with Ice 3.3 on RHEL 4 and apparently Debian Etch, you need this /usr/lib/nptl all the time: when building Ice, and also when linking your own Ice-based applications.

Cheers,
Bernard

jonaf1

Thanks for the reply, Bernard. Then again, thanks for the reiteration of what you originally stated in post #2; I just realized that your first answer was the answer I needed but failed to "decipher" into "-L/usr/lib/nptl -lpthread" command arguments.

jonaf1

Hint for others: in reality all you have to do is add /usr/lib/nptl to your LD_LIBRARY_PATH and you won't have to touch the actual source.

However, I'm having another problem when trying to run icebox:

07/07/08 14:05:36.547 /opt/Ice-3.3/bin/icebox: error: 
ServiceManager: unable to load entry point 
`IceStormService,32:createIceStorm': libIceStormService.so.32: cannot 
open shared object file: No such file or directory

I've removed all Ice 3.2 stuff totally and recompiled (from a fresh tgz extraction) and it still is looking for the 3.2 library. There's no occurrences of 3.2 or so.32 in any of the binary files or source files for that matter.

Is this problem something I would fix at the link or runtime stage, and how would I do so?

bernard

Hi Jonaf,

You just need to edit your Ice configuration file (or IceGrid deployment, if you're using IceGrid), and replace "IceStormService,32" by "IceStormService,33".

Cheers,
Bernard

jonaf1

Sometimes I make my life much more complicated than it needs to be....8 compilations later and it was just a config file change...lol. Thanks, Bernard, for the answer.

desrod

Ice 3.3.1 does not build on RHEL3 or RHEL4

I've been trying to build Ice 3.3.0 or 3.3.1 on our RHEL3 and RHEL4 machines in support of some internal developers who are testing it, and running into no end of problems.

The reason I'm posting in this thread, is because the errors I'm getting are described here. Here's what I'm seeing:

c++ -c -I. -I../../include -m64 -ftemplate-depth-128 -Wall -D_REENTRANT -I/usr/include/nptl -fPIC -O2 -DNDEBUG Gen.cpp
c++ -c -I. -I../../include -m64 -ftemplate-depth-128 -Wall -D_REENTRANT -I/usr/include/nptl -fPIC -O2 -DNDEBUG Main.cpp
rm -f ../../bin/slice2cpp
c++ -Wl,--enable-new-dtags -Wl,-rpath,/path/to/Ice/3.3.1-ml01/lib64 -m64 -ftemplate-depth-128 -Wall -D_REENTRANT -I/usr/include/nptl -fPIC -O2 -DNDEBUG -L../../lib  -o ../../bin/slice2cpp Gen.o Main.o -lSlice -lIceUtil -L/usr/lib64/nptl -lpthread -lrt -Wl,-rpath-link,/path/to/mcpp/2.7.2/lib64
../../lib/libIceUtil.so: undefined reference to `pthread_condattr_setclock'
collect2: ld returned 1 exit status
make[6]: *** [../../bin/slice2cpp] Error 1

But RHEL3 and RHEL4 do not have /usr/lib/nptl or /usr/lib64/nptl at all, in any package; upstream or non.The headers are there, but no libraries exist for these distributions by Red Hat.

I suspect, from further reading in this thread and a few others, that this flat-out isn't going to build at all on these platforms, even if I was to shoehorn the NPTL libraries onto the machine, is it?

So what's next? Where do I go from here?

bernard

Hi David,

Welcome to our forums!

We provide Ice 3.3.1 RPMs for RHEL4, so the simplest is to download and install these RPMs. See: http://www.zeroc.com/download.html#linux_rpms

If you prefer, you can also build Ice 3.3.1 on RHEL4 (that's just a little more work). Please read the INSTALL.LINUX in the source distribution; you'll notice:

Required packages
-----------------

On RHEL 4, Ice requires the nptl-devel package, which contains the
function pthread_condattr_setclock.

(You will also need this nptl-devel RPM for any C++ development using Ice)

We don't support RHEL3 with Ice 3.3.1, so this will definitely require more work. I would start by looking at this thread: http://www.zeroc.com/forums/help-center/3900-building-ice-3-3-0-red-hat-9-a.html

Best regards,
Bernard

desrod

bernard wrote: »

We provide Ice 3.3.1 RPMs for RHEL4, so the simplest is to download and install these RPMs.

Unfortunately, we can't install RPMs into our global namespace, they're just not compatible (we can do SRPM, but then we're back where we started, because those have to be compiled).

If you prefer, you can also build Ice 3.3.1 on RHEL4 (that's just a little more work).

RH 4AS-6.1 is where I'm currently getting the errors I posted above. If you do not support RHEL3, that's better for me... one less platform I have to support internally

(You will also need this nptl-devel RPM for any C++ development using Ice)

Apparently that doesn't exist for RH 4AS-6.1 that anyone internally that administers these machines can find. We're still looking though...

bernard

Hi David,

nptl-devel is a standard RPM included in all RHEL 4 Advanced Server distributions.

For example, here is a fairly recent RedHat errata that includes an updated nptl-devel for RHEL 4: rhn.redhat.com | Red Hat Support

Best regards,
Bernard