Home Help Center

Application add is very slow, what happened?

eaglecneaglecn Member Yong WANGOrganization: ICT,chinaProject: DBroker
Hi,
We use icegrid to manage several applications, find that adding some an application is slow. So what happened when adding an application or how can we trace these activities?
thanks!

Comments

  • benoitbenoit Rennes, FranceAdministrators, ZeroC Staff Benoit FoucherOrganization: ZeroC, Inc.Project: Ice ZeroC Staff
    Hi,

    You can add the following traces on the registry:

    IceGrid.Registry.Trace.Server=3
    IceGrid.Registry.Trace.Object=1
    IceGrid.Registry.Trace.Adapter=1

    This should trace the newly added objects, adapters and servers and the deployment of each server on the nodes.

    Cheers,
    Benoit.
  • eaglecneaglecn Member Yong WANGOrganization: ICT,chinaProject: DBroker
    I add cout code to function of "IceProxy::IceGrid::Admin::addApplication" in file of Ice-3.2.1/src/IceGrid/Admin.cpp just now as following,
                cout<<"before checktwoway=="<<endl;
                __checkTwowayOnly(__IceGrid__Admin__addApplication_name);
                cout<<"before getDelegate=="<<endl;
                __delBase = __getDelegate();
                cout<<"before dynamic_cast=="<<endl;
                ::IceDelegate::IceGrid::Admin* __del = dynamic_cast< ::IceDelegate::IceGrid::Admin*>(__delBase.get());
                cout<<"before _del->addApp=="<<endl;
                __del->addApplication(descriptor, __ctx);
    
    and find no exception.

    After tried several times, the same symptom reappears, namely, the add application command line is hanging.

    this time, it reports
    "IceGrid.Registry.Client.ThreadPool' is running low on threads" frequently,

    I got no idea with this parameter.
    after checking, the registry process has more than 18 threads.

    where is wrong?
  • benoitbenoit Rennes, FranceAdministrators, ZeroC Staff Benoit FoucherOrganization: ZeroC, Inc.Project: Ice ZeroC Staff
    Sorry, without more information it's impossible to say what could be wrong. Did you try adding the tracing suggested in my previous post? Can you copy/paste the tracing output here?

    Cheers,
    Benoit.
  • eaglecneaglecn Member Yong WANGOrganization: ICT,chinaProject: DBroker
    The following info is all I got from a rsh terminal, is it enough?
    the last application need 4 minutes to be loaded, while others almost need no time.

    [ 01/13/09 17:03:58.078 Adapter: added adapter `ddl_yf-softnode3.DDLAdapter' ]
    [ 01/13/09 17:03:58.079 Object: added object `ddl_yf-softnode3' ]
    [ 01/13/09 17:03:58.079 Server: added server `ddl_yf-softnode3' (`2C521B8F-2B06-4146-A121-4A128D8B8903', `1') ]
    [ 01/13/09 17:03:58.079 Server: loading `ddl_yf-softnode3' on node `yf-softnode3' ]
    [ 01/13/09 17:03:58.081 Server: loaded `ddl_yf-softnode3' on node `yf-softnode3' ]
    before checktwoway==
    before getDelegate==
    before dynamic_cast==
    before _del->addApp==
    [ 01/13/09 17:03:58.117 Adapter: added adapter `load_YF-softnode4.LoadAdapter' ]
    [ 01/13/09 17:03:58.117 Object: added object `load_YF-softnode4' ]
    [ 01/13/09 17:03:58.117 Server: added server `load_YF-softnode4' (`95E14DCE-AF6B-47D0-9515-BB0F952E0903', `1') ]
    [ 01/13/09 17:03:58.117 Server: loading `load_YF-softnode4' on node `YF-softnode4' ]
    [[email protected] script]$ [ 01/13/09 17:07:07.092 Server: loaded `load_YF-softnode4' on node `YF-softnode4' ]

    and only this application locates at the other server
  • benoitbenoit Rennes, FranceAdministrators, ZeroC Staff Benoit FoucherOrganization: ZeroC, Inc.Project: Ice ZeroC Staff
    From the traces, it appears that the server loading on YF-softnode4 hangs since you're not getting the "Server: loaded `load_YF-softnode4' on node `YF-softnode4'" message. Is the host where YF-softnode4 runs accessible from the registry host?

    Also, you should add timeouts to the IceGrid registry and node endpoints to avoid requests to block for too long if something goes wrong. See the configuration files from cpp/demo/IceGrid/replication for an example on how to set timeouts.

    Cheers,
    Benoit.
  • eaglecneaglecn Member Yong WANGOrganization: ICT,chinaProject: DBroker
    these machines can access each other and not all loading of this application are not successful. In fact, adding process is slow but can be done finally.

    So, I am wonder during the hanging, what operation is ongoing? Is it just try to accessing remote server?

    Why other applications never have this problem? Processing properites in program has relation with it? But I think no processes are to be started during that time.
  • eaglecneaglecn Member Yong WANGOrganization: ICT,chinaProject: DBroker
    I got the following message when removing this application,
    [ 01/13/09 17:21:51.357 Server: couldn't unload `load_YF-softnode4' on node `YF-softnode4':
      Network.cpp:624: Ice::ConnectTimeoutException:
      timeout while establishing a connection ]
    
    [ 01/13/09 17:53:43.501 Server: couldn't unload `load_YF-softnode4' on node `YF-softnode4':
      Network.cpp:664: Ice::ConnectionRefusedException:
      connection refused: Connection refused ]
    
    why it is timout to establish a connection?
  • benoitbenoit Rennes, FranceAdministrators, ZeroC Staff Benoit FoucherOrganization: ZeroC, Inc.Project: Ice ZeroC Staff
    For some reasons, the registry can't contact the node YF-softnode4. You could try adding network tracing on the IceGrid registry with Ice.Trace.Network=2 to figure out on which IP address the registry tries to contact the node.

    If you didn't specify any host with the -h HOST option in the IceGrid node endpoints, it's possible that the registry tries to connect to an unreachable IP address if the host where the node is running has multiple network interfaces.

    You can try to specify explicitly the IP address in the node endpoints, for example:
       IceGrid.Node.Endpoints=tcp -h 192.168.1.2
    

    See 28.4.6 Endpoints in the Ice manual for more information on object adapter endpoints.

    Cheers,
    Benoit.
  • eaglecneaglecn Member Yong WANGOrganization: ICT,chinaProject: DBroker
    I believe the problem is solved after i set the icegridnode configure file.
    thanks a lot!
Sign In or Register to comment.