Icebox service times out when activated

jcbellonjcbellon Member BellonOrganization: J. Safra Sarasin BankProject: Grid Computing

I did set up a DLL to run with Icebox embedding some third party library and loading some third-party DLL.
When Icebox is started it does not crash but stall before entering the entry point ( the "create" function that returns an implementation instance of the service class). The problem could be the initialization of these third-party DLL but I was wondering how to detect where the problem lies ? Since I cannot get a grip on the running code (attaching process to Visual Studio) because my code doesn't seems to be invoked at all I do not know how to identify the faulty part.

Answers

  • benoitbenoit Rennes, FranceAdministrators, ZeroC Staff Benoit FoucherOrganization: ZeroC, Inc.Project: Ice ZeroC Staff

    Hi,

    Why can't you attach to the running process with Visual Studio to check where the hang occurs? Even if it's not yet running any of your code, it could provide some valuable information. Otherwise, you could first try to get your third-party DLL working with a regular executable that also links with the Ice DLL and see if this works.

    Cheers,
    Benoit.

  • jcbellonjcbellon Member BellonOrganization: J. Safra Sarasin BankProject: Grid Computing

    Thanks for the answer.

    The thing is that I can attach the process but as I cannot break in the code, I get no call stack information for example or current state, I do not know how I can extract interesting information at this stage.

    But I am gonna try to build my own exe emulating icebox and let you know my findings !

    Regards,

    JC

  • xdmxdm La Coruña, SpainAdministrators, ZeroC Staff Jose Gutierrez de la ConchaOrganization: ZeroC, Inc.Project: Ice Developer ZeroC Staff

    Hi,

    Did you try to configure the ZeroC symbols server? see. You might also want to try with a Debug build, don't forget to use the icebox "Debug" executable for "Debug" builds.

  • jcbellonjcbellon Member BellonOrganization: J. Safra Sarasin BankProject: Grid Computing

    Thanks for the Symbol Server tip but unfortunately such online settings are not really possible within the network of the company. Would it be downloadable as a file and offline installable ?

  • jcbellonjcbellon Member BellonOrganization: J. Safra Sarasin BankProject: Grid Computing
    edited January 5

    I did a program that tries to emulate the icebox.exe in a very simple way, it does calls the create function for the exported function of the DLL and start the service. That this program works whereas icebox.exe is hanging before performing this task.

    #include <Ice/Ice.h>
    
    #include <IceBox/IceBox.h>
    #undef UNICODE 
    #include <windows.h>
    
    using namespace std;
    
    typedef IceBox::Service* (__stdcall *f_funci)(Ice::CommunicatorPtr);
    
    class Server : public Ice::Application
    {
    public:
        virtual int run(int argc, char* argv[]);
    };
    
    int main(int argc, char* argv[])
    {
        cout << "Starting" << endl;
        Server app;
            int status = app.main(argc, argv, "C:\\IceGrid\\Node\\Data\\servers\\IceGrid.ServiceToTest\\Config\\config_IceGrid.ServiceToTest");
            cout << "status" << status << endl;
            return status;
    
        return EXIT_FAILURE;
    }
    
    int Server::run(int argc, char*[])
    {
        HINSTANCE hGetProcIDDLL = LoadLibrary(L"C:\\IceGrid\\Node\\Data\\servers\\IceGrid.ServiceToTest\\distrib\\ServiceToTest\\x64\\ServiceToTest.dll");
        if (!hGetProcIDDLL) {
            std::cout << "could not load the dynamic library with error " << GetLastError() << std::endl;
            return EXIT_FAILURE;
        }
    
        f_funci funci = (f_funci)GetProcAddress(hGetProcIDDLL, "create");
        if (!funci) {
            std::cout << "could not locate the function" << std::endl;
            return EXIT_FAILURE;
        }
    
        IceBox::Service* serv = funci(communicator());
        try {
            serv->start("ServiceToTest", communicator(), Ice::StringSeq());
        }
        catch (exception& e) {
            cerr << "Error To determine " << e.what() << endl;
            cout << "failure" << endl;
        }
        communicator()->waitForShutdown();
    
        FreeLibrary(hGetProcIDDLL);
    
        return EXIT_SUCCESS;
    }
    
  • jcbellonjcbellon Member BellonOrganization: J. Safra Sarasin BankProject: Grid Computing

    The icebox.exe is hanging. I could attach it to VS2013, pause the program and look at the call stack. whenever I pause I always end up in the same disassembled line of the 3rd-party DLL. This before DLLMain and IceBox::Service* create(CommunicatorPtr) are called on my own DLL.

    This 3rd party DLL, during initialization according its editor, is preliminary checking some license files and I have no access to source code or debug files to identify the issue there. Could it be that icebox.exe is modifying the surrounding execution context of the DLL preventing it to perform correctly its own initialization task ? For example standard output is redirected but could it be something else that could conflicts with it ?

  • benoitbenoit Rennes, FranceAdministrators, ZeroC Staff Benoit FoucherOrganization: ZeroC, Inc.Project: Ice ZeroC Staff

    Hi,

    IceBox is implemented as an Ice::Service, see https://github.com/zeroc-ice/ice/blob/master/cpp/src/IceBox/Service.cpp

    You could try changing your test executable implementation to use Ice::Service instead of Ice::Application and see if this makes a difference.

    Cheers,
    Benoit.

  • jcbellonjcbellon Member BellonOrganization: J. Safra Sarasin BankProject: Grid Computing

    Thanks Benoit for the tip !

    Currently I manage to workaround the problem by using Linker Option /DELAYLOAD on the faulty DLL. This a workaround, a not a solution. In the meantime I work with the Third Party DLL software developers to try to identify the issue. I will post our findings if any relevant,

    I did took a look to the IceBox implementation on github but it seems that I need virtually to clone the Icebox Implementation to mimic the behavior. The advantage is that would give me perhaps a change to reach a breakpoint before hanging but this is not even sure.

    I continue searching because this issue is a showstopper for the project if we cannot make live together Icebox and the 3rd party Library I wish to wrap for grid usage.

    Thanks a lot trying constantly to bring answers to your library users ! Ice is a great tool I do really promote it because it deserve to be known.

    Regards,

    Jean-Christophe

  • jcbellonjcbellon Member BellonOrganization: J. Safra Sarasin BankProject: Grid Computing
    edited January 10

    Hello,

    In the deep examination of the DLLs loaded at run-time, I've discovered that msvcrt.dll is used at the same time than mcvcr120.dll (vs2013). And when I looked why this dll was loaded (I originally though I used a wrong lib) I found out that all Ice executables in the Ice VS21013 NuGet package (icebox.exe, icegridnode.exe, ...) have the same dependency. Actually it look like it is a dependency of Ice37.dll itself. Isn't it something that could lead to heap corruption ?

    This could be a lead for solving the nasty problem I experience. It is probably unrelated but it is worth checking in my opinion.

    Kind regards,

    JC

  • xdmxdm La Coruña, SpainAdministrators, ZeroC Staff Jose Gutierrez de la ConchaOrganization: ZeroC, Inc.Project: Ice Developer ZeroC Staff

    Hi,

    Ice and IceBox vs2013 does not have any dependency on msvcrt.dll

    see:

    dumpbin /DEPENDENTS packages\zeroc.ice.v120.3.7.0\build\native\bin\Win32\Release\icebox.exe
    Microsoft (R) COFF/PE Dumper Version 14.12.25834.0
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    
    Dump of file packages\zeroc.ice.v120.3.7.0\build\native\bin\Win32\Release\icebox.exe
    
    File Type: EXECUTABLE IMAGE
    
      Image has the following dependencies:
    
        MSVCP120.dll
        ice37.dll
        icebox37.dll
        MSVCR120.dll
        KERNEL32.dll
    
      Summary
    
            1000 .data
            8000 .rdata
            2000 .reloc
            1000 .rsrc
           10000 .text
    
    dumpbin /DEPENDENTS packages\zeroc.ice.v120.3.7.0\build\native\bin\Win32\Release\ice37.dll
    Microsoft (R) COFF/PE Dumper Version 14.12.25834.0
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    
    Dump of file packages\zeroc.ice.v120.3.7.0\build\native\bin\Win32\Release\ice37.dll
    
    File Type: DLL
    
      Image has the following dependencies:
    
        bzip2.dll
        ADVAPI32.dll
        WS2_32.dll
        IPHLPAPI.DLL
        RPCRT4.dll
        dbghelp.dll
        SHLWAPI.dll
        MSVCP120.dll
        MSVCR120.dll
        KERNEL32.dll
    
      Summary
    
           13000 .data
           A6000 .rdata
           1D000 .reloc
            1000 .rsrc
          176000 .text
    
  • xdmxdm La Coruña, SpainAdministrators, ZeroC Staff Jose Gutierrez de la ConchaOrganization: ZeroC, Inc.Project: Ice Developer ZeroC Staff

    Actually ICE37.dll imports ADVAPI32.dll and ADVAPI32.dll imports MSVCRT.dll but that shound't be a problem

  • jcbellonjcbellon Member BellonOrganization: J. Safra Sarasin BankProject: Grid Computing

    Thanks for having this checked so quickly.
    I will continue my investigations then.

  • xdmxdm La Coruña, SpainAdministrators, ZeroC Staff Jose Gutierrez de la ConchaOrganization: ZeroC, Inc.Project: Ice Developer ZeroC Staff

    Hi,

    Can you clarify what you are trying to do? Did you try to create the server as an Ice::Application and directly link with your thirdparty DLL rather than dynamic load it or that is non an option for your use case?

  • jcbellonjcbellon Member BellonOrganization: J. Safra Sarasin BankProject: Grid Computing

    Hi xdm,

    I tried to reproduce an exe that would hang when loading my own DLL made for icebox.exe, so I can understand why it does freeze before entering my code at DLL load time.

    In parallel I am working with the 3rd party DLL editor since it does freeze in their DLL initialization waiting for some shared resources. One noticeable finding is the problem only exists in release mode. The 3rd-Party DLL compiled in debug mode does not show any issue when loading.

    I did use an Ice::Application with a dynamic load of the DLL because it did guess icebox would use the same technique as it cannot statically link with the DLL created by the customer. But for some reason my first attempt did work correctly with my own exe. So I need to come closer to the implementation of the icebox.exe and potentially rebuilt it so I can access symbols in release mode (I cannot bind your symbol sever because of the company proxy settings).

    Regards,

    JC

  • benoitbenoit Rennes, FranceAdministrators, ZeroC Staff Benoit FoucherOrganization: ZeroC, Inc.Project: Ice ZeroC Staff

    Hi Jean Christophe,

    Did you try replacing Ice::Service with Ice::Application in your test app? That's really the only major difference I can think of between IceBox and your test app.

    In any case, it sounds like your making good progress to get to the bottom of this. The easiest way to figure out the problem would be to get a stack dump of the hang, for this you need the PDBs of the release 3rd party DLL.

    Also note that IceBox isn't a requirement for running Ice services on IceGrid... you could also very well just implement your service as a regular executable linked with the 3rd party DLL if this issue can't easily be worked around.

    Cheers,
    Benoit.

Sign In or Register to comment.