Archived

This forum has been archived. Please start a new discussion on GitHub.

Getting IceE demos running on Arm-Linux

I've spent most of the day trying to figure out how to move the Ice client I've written on my Windows machine over to one of my many Arm boards (BeagleBoards and Utilite Pros), but have hit something of a dead end. I was able to port the code over to x86 Linux and get everything running there (after fighting through all kinds of bugs with GCC 4.7+Ice). Next, I downloaded and built IceE on my Linux development computer using the arm-linux-gnueabi-g++-4.7 toolchain, which seemed to work and produced an ELF 32-bit LSB ARM EABI5 executable, exactly as I would expect and need for my target platform. I grabbed the entire IceE folder and moved it over to my BeagleBoard and ran 'make install', which seemed to work, as far as I could tell.

However, when I tried to run cppe/demo/IceE/hello/server, I got the complaint:

"error while loading shared libraries: libIceE.so.13: cannot open shared object file: No such file or directory"

All of the shared libraries are definitely present, and I've tried various ways of building test projects (outside of the demo projects in the IceE folder, I also built a test project in Eclipse using the same toolchain where I directly included all of the shared object files in the build folder with the executable). I have a feeling that I am just doing something wrong in general.

Are there some steps that I have missed? What is the general process for cross-compiling for and deploying to an ARM target? Any help/guidance/pointing-out-of-my-obvious-mistakes would be greatly appreciated. I am primarily a Windows developer, so most of the Linux/ARM/cross-compiling activities are somewhat new territory for me.

Thanks

Comments

  • xdm
    xdm La Coruña, Spain
    Hi,

    The most likely cause for this error is that the library ins't installed in a directory known by ld.

    You can add the directory containing IceE libs to your LD_LIBRARY_PATH environment variable, or you can install IceE libraries in a directory already known by ld like "/usr/lib", see ldconfig man page for details.
  • I tried doing this, but nothing seems to have changed. Same error message about missing libIceE.so.13.

    "make install" installed the libs to /opt/IceE-1.3.0/libs, so I added

    export LD_LIBRARY_PATH="/opt/IceE-1.3.0/libs"

    to my .bashrc file. I cd'd to that location and checked the file info for the .so's. libIceE.so.1.3.0 is a proper 32bit ARM shared library and libIceE.so.13 is correctly symlinked to that. I cross compiled some other applications using the same toolchain and methodology, so I'm not really sure what could be going wrong.

    I appreciate your help with trying to figure this out.
  • xdm
    xdm La Coruña, Spain
    Libraries are usually installed to "/opt/IceE-1.3.0/lib" not to "/opt/IceE-1.3.0/libs", note the extra 's' at the end, maybe you have a typo?
  • That was a typo in the post only, apologies.
  • Here is a step by step guide describing my entire process

    1. Download IceE-1.3.0
    2. Extract tar file
    3. Navigate to IceE-1.3.0/cppe/config/
    4. Make following changes in Make.rules:
    a. L:142 & L:145 - Add -fpermissive
    5. Make following changes in Make.rules.Linux
    a. Change the tools listed as "c++" and "ar" to their "arm-linux-gnueabi-" equivalents (g++-4.7 and ar)
    b. Comment out machine specific instructions on lines 32-66 (anything relating to x86 / x86_64, SPARC, etc)
    6. Repeat similar changes made in 4. and 5. for the files in IceE-1.3.0/cpp/config/
    7. make configure
    8. make
    9. Copy entire IceE-1.3.0 folder over to ARM Linux board /home/user/ directory
    10. Open terminal in IceE-1.3.0 (logged in as root)
    11. make install
    12. Verify /opt/IceE-1.3.0/ was created and the lib files are present
    13. Verify libIceE.so.1.3.0 is a 32bit ARM dynamically linked shared object
    14. Verify libIceE.so.13 is a symlinked to libIceE.so.1.3.0
    15. Verify that export LD_LIBRARY_PATH="/opt/IceE-1.3.0/lib" is present in .bashrc
    16. Navigate to IceE-1.3.0/cppe/demo/IceE/hello/server
    16. ./server
    17. error while loading shared libraries: libIceE.so.13: cannot open shared object file: No such file or directory
  • xdm
    xdm La Coruña, Spain
    What is the output of this commands in your arm linux board:
    echo $LD_LIBRARY_PATH
    ldd server
    ldd /opt/IceE-1.3.0/lib/libIceE.so.13
    
  • /opt/IceE-1.3.0/lib
    not a dynamic executable
    not a dynamic executable

    For arm executables, I have generally always had to use readelf -d <file> to get that information.

    The readelf output for server:
    #9299571 - Pastie

    For libIceE.so.13:
    #9299574 - Pastie
  • Whoa, in both of those:

    0x0000001d (RUNPATH) Library runpath: [/opt/IceE-1.3.0/lib64]

    This might be a problem. I'm not sure where that originates from though. Any thoughts?
  • xdm
    xdm La Coruña, Spain
    This come from cppe/config/Make.rules.linux is the -rpath option in ld platform flags
    LDPLATFORMFLAGS 	= -Wl,--enable-new-dtags -Wl,-rpath,$(install_libdir)
    

    You can try to copy lib to lib64 to see if that solves the problem.

    When you cross-compile in x64 you can force 32bit build setting, LP64=no, in cppe/config/Make.rules
  • In a naive attempt for a quick fix, I renamed the /opt/IceE-1.3.0/lib folder to /lib64/ and changed my LD_LIBRARY_PATH to match it, hoping it would resolve after that (which, unfortunately and expectedly, it did not). I don't think the arm-linux-gnueabi- toolchain is even capable of producing output files that are 64-bit, so I question whether or not this could be a true incompatibility between architectures.
  • xdm
    xdm La Coruña, Spain
    I can be wrong, but I get the impression that the libIceE.so is failing to because ld-linux.so.3 dependency, do you have this library in your arm board?

    What Linux version do you rum in you ARM board? See here for a similar problem with Ubuntu ARM No such file or directory error
  • I ran into this problem when I was trying to run a cross compiled "hello world" application before moving on to trying to cross compile IceE applications. I solved it using the same thread that you just linked, so I've already copied ld-linux.so.3 to the appropriate folder, though you are right in acknowledging that the symptoms of the problem seem to be exactly the same.

    As for Linux versions, I am running 14.04 on one and 12.04 on another.
  • xdm
    xdm La Coruña, Spain
    What about libpthread.so, i guess it could cause the same problems
  • Yea, I just had the same idea, so I checked the location of all of the other dependencies listed in the readelf for both server and libIceE.so.13, and they were in fact all in the /lib/arm-linux-gnueabihf/ directory, just like ld-linux.so.3 had been. I copied everything in that folder up into /lib/, but the result has not changed, unfortunately.
  • Today, I switched over to trying some of the things we discussed yesterday on my Utilite Pro, which is running Ubuntu 12.04 armel (as opposed to the BeagleBoard which is running 14.04 armhf).

    I copied the fully built IceE folder over to the board and ran 'make install'. Tried to run the Hello 'server' and was greeted with our familiar friend, 'no such file' for IceE.so.13. I remembered that I needed to change the lib folder's name to lib64, so I did that (just as I had tried on the other board), and tried to run server again. This time I was greeted with a new message!

    ./server: /lib/arm-linux-gnueabi/libc.so.6: version 'GLIBC_2.17' not found (required by /opt/IceE-1.3.0/lib64/libIceE.so.13)

    Did some searching on my board and found that I have libc-2.15 installed, so I searched google to see if I could install 2.17. This is where things get tricky. There are no 2.17 releases (that I could find) for Ubuntu 12.04 armel or armhf (I have the option to change to armhf if I need to), however, there are 2.17 releases for Raring, Saucy, and Trusty. Unfortunately, I can't upgrade the Ubuntu version on the Utilite any further than it is now, as the custom kernel for the Utilite board (which is required for operation) does not support their usage yet and there is no word on them upgrading it any time soon.

    Being a Linux newbie of sorts, I feel like there might be a way to build IceE with an older version of libc (2.15), but I'm not sure how to go about trying that. Any thoughts?
  • xdm
    xdm La Coruña, Spain
    Seems to me that you are not using the right toolchains:

    For 12.04 armel I think you should be using, "arm-linux-gnueabi-g++-4.6" this toolchain comes with libc 2.15.

    For your BeagleBoard armhf with Ubuntu 14.04 i think the correct toolchain is "arm-linux-gnueabihf-g++-4.8", note the "gnueabihf" suffix instead of "gnueabi".
  • Odd, because I've built simple hello world applications that ran properly for both boards using the same arm-linux-gnueabi- toolchain. However, your suggestion is the best thing I've got to go on right now, so I'll give that a shot. Thanks.
  • Xdm, you're a saint. IceE is now working flawlessly on the BeagleBoard having compiled with the arm-linux-gnueabihf toolchain. Still don't understand why the other toolchain worked for some things but not this, but I'm not complaining, now that I have a working solution, haha.

    An article concerning beagleBone development also backed up your statement:

    "- Graemefisheratwork let me know that he has found that when using the ubuntu armhf distros, applications should be cross-compiled using arm-linux-gnueabihf- and not arm-linux-gnueabi-. This seems to have worked for him on the ubuntu 12.04 armhf build."

    I think what I'll do for the Utilite board is switch it over to 12.04 armhf so I can keep things consistent in my build process.

    Thanks again!
  • xdm
    xdm La Coruña, Spain
    I'm glad it's working