Icegrid for dummies

b1c

Hi all!

Does anyone have or come across any blogs, posts or something similar showing how to deploy ICE (especially icegrid) in a manner accessible to audience outside the technologists' community. I have a computationally intensive job to do (in Python) that could easily take a few days to finish and I would like to use icegrid so that other computers could undertake parts of it. I think this is something that fits quite well within zeroc ice's ecology. The problem is that I haven't managed to get the slightest grip on it. Ok, i can run the (python) demos that the ice people have made, I tried to understand (in vain) the manual/articles on the site and I have installed icegridgui but I still dont know basic things, like for example:

How to 'put' computers on the grid
and
Make workers (slave pcs) pick up and perform a task and return back the result

I am pretty comfortable with Python but I have zero knowledge of C/C++. I should be able to do some small edits to existing Java code. Is this beyond my reach then?

Any help highly appreciated

Thanks!

benoit

Hi,

I recommend to first familiarize yourself with Ice before looking into IceGrid.

Ice a middleware library that allows a client and server or multiple processes (possibly written in different programing languages) to easily interact between each other through a well-defined contract specified with the Slice language.

To perform computation on many nodes, you will need to run a server process on each node where each process listens on a given TCP/IP port and exposes a Slice interface that you define (it can also be multiple interfaces). This Slice interface can have multiple operations to perform the different the tasks you need. The Ice client will talk to these Ice servers through this Slice interface to perform some computation.

Next, once you have defined this Slice interface and implemented the client and server, you can look into IceGrid. IceGrid mostly provides two services that will help you to deploy your application at a larger scale: a discovery/lookup service and a deployment service.

The discovery/lookup service will allow to easily find servers, you will not need to know the IP and TCP/IP port number to contact the server (you can view this service like the DNS service for the Internet). The deployment service allows to deploy the severs on the different nodes and handle the launch and termination of the server processes hosted by IceGrid.

Here are few pointers to our manual and technical articles that should help better understanding how Ice works:

• https://doc.zeroc.com/ice/3.7/introduction
• https://doc.zeroc.com/technical-articles/general-topics/chat-demo
• https://doc.zeroc.com/technical-articles/icegrid-articles/teach-yourself-icegrid-in-10-minutes

Let us know if you need additional information on this!

Cheers,
Benoit.

bernard

Hi George,

With IceGrid, the "distribute job" model is typically like Benoit described: a client divides up a large job in smaller computations and send them for processing on various servers started and managed by IceGrid.

With your model:

Make workers (slave pcs) pick up and perform a task and return back the result

you can use Ice, but you may not need IceGrid at all. The workers could be Ice clients that "pick-up" (request) work from an Ice server and then return the result to that Ice server. The resulting Slice (implemented by your one server) could look like:

module Probe
{
        interface JobResult
        {
             void done(string output);
        }

        struct Job
        {
             string input;
             JobResult* result;
        }

        interface JobRegistry
        {            
             Job getNewJob();
        }
}

Each worker would call getNewJob on the well-known JobRegistry object, and after processing would call done on the JobResult object. No need for IceGrid here.

Best regards,
Bernard

b1c

Thanks a lot Benoit and Bernard. I will try to put together a really basic example, a childish one and see how it goes from there,

Best regards
George