Archived

This forum has been archived. Please start a new discussion on GitHub.

filesystem implementation/ followup to my previous question

Hi again,

in my previous question I was asking for more detailed adapter statistics.

My goal is to implement something similar to the filesystem example in the Ice documentation.

To do so, I suppose that using a ServantLocator makes sense to be able to re-use the servants that I returned in calls like

File* getFile(string name);

for other requests for the same File.
But consecutive calls to getFile() for retrieving different files will result in increasing numbers of servants being instantiated.
Is it correct that I understand ServantLocators as some kind of servant cache? If so, are there any hints on how to limit the number of servants that are held by such a locator?
Generating access statistics like LRU and LFU is a pattern that's easy to understand and to implement, but how shall I clean up the cache?

E.g. would you suggest to use a thread that checks the cache from time to time, sees if there are too many servants, locks the servant array and removes servants that aren't currently needed?

Do different approaches exist?
Respectively: Does anyone have experience with using ServantLocators?

regs,

Stephan

Comments

  • marc
    marc Florida
    I would recommend to use one single servant for all Ice objects representing files. That's called a default servant. Have a look at chapter 16.7.2 in the Ice documentation ("Default Servants").

    For a more elaborate example of a file transfer service, check out the code of IcePatch. It also uses default servants.

    You could also use an evictor, but in this case, this won't give you much benefit, but it would make your application more complicated. Have a look at chapter 16.7.4 for more information ("Servant Evictors"), or at the demo in demo/book/evictor.
  • Originally posted by marc
    I would recommend to use one single servant for all Ice objects representing files. That's called a default servant. Have a look at chapter 16.7.2 in the Ice documentation ("Default Servants").

    I was also thinking about this as an alternative.
    Can you do me (as an almost-newbie :) ) a hint concerning this modified pseudo code from page 404f:

    FilePrx DirectoryI::getFile(const std::string name, const Ice::Current& current) const
    {
    // have to retrieve file proxy from locator here
    }

    ?

    And, the second question that arises, what if I want to return a list of files in case I list a directory. Does it make sense to do this

    FilePrxSequence DirectoryI::list(...)

    If I only have one file servant, this should be quite difficult, or did I miss something?

    Thanks,

    Stephan
  • marc
    marc Florida
    Originally posted by stephan
    I was also thinking about this as an alternative.
    Can you do me (as an almost-newbie :) ) a hint concerning this modified pseudo code from page 404f:

    FilePrx DirectoryI::getFile(const std::string name, const Ice::Current& current) const
    {
    // have to retrieve file proxy from locator here
    }

    You simply call ObjectAdapter::createProxy to create a proxy for the Ice object representing the file. For example, if the object identity contains the path of the file, you can use:

    FilePrx file = FilePrx::uncheckedCast(_adapter->createProxy(... identity constructed from file path ...));

    Again, please have a look at the IcePatch code (file IcePatchI.cpp) for an example.
    Originally posted by stephan
    And, the second question that arises, what if I want to return a list of files in case I list a directory. Does it make sense to do this

    FilePrxSequence DirectoryI::list(...)

    If I only have one file servant, this should be quite difficult, or did I miss something?

    You should have two default servants, one for files, and one for directories. Your servant locator can return the correct servant based on the Ice object identity. See IcePatch for an example :)
  • Hi again!

    I took a look at IcePatch now :)

    You should have two default servants, one for files, and one for directories. Your servant locator can return the correct servant based on the Ice object identity. See IcePatch for an example :)

    I saw that the Directory method

    nonmutating FileDescSeq getContents();

    does not actually return a number of File proxies but a number of File descriptors.
    This results in two classes that contain file information, FileDesc and File which doesn't seem the cleanest possible solution.
    You seem to use FileDesc as a kind of bootstrap for retrieving the 'real' file information (it only contains a md5 code). Is that correct?

    imho, a cleaner solution would be a method like this

    nonmutating FilePrxSeq getContents();

    with
    sequence<File*> FilePrxSeq;

    This would result in faster access time when listing directory contents.

    However I assume that this is not possible with a singleton servant.

    regs,

    Stephan
  • marc
    marc Florida
    That's not correct. The FileDesc contains the md5, as well as the proxy to the file object. (Derived from FileDesc, you have RegularDesc and DirectoryDesc, which contain a RegularPrx or FilePrx, respectively.)

    For IcePatch, this method saves several operation calls. If you would only return a FilePrx, you would need one additional operation call to get the md5, and another additional operation call for the checkedCast to find out if the FilePrx is a RegularPrx or a DirectoryPrx.

    The above has nothing to do with singleton approach or not. Both methods work with a singleton approach.
  • Hi again!

    For IcePatch, this method saves several operation calls. If you would only return a FilePrx, you would need one additional operation call to get the md5, and another additional operation call for the checkedCast to find out if the FilePrx is a RegularPrx or a DirectoryPrx.

    I hope that this is my last question :) and I don't run no flame war here :)

    However, thanks for the kind answers, it really helps a lot in understanding how you are using your own framework!

    What I'm asking myself is why you didn't add a md5 method to the interface but instead create a separate class for holding the md5. This is no criticism I'm only curious to know more about your design decisions.

    regs,

    Stephan
  • marc
    marc Florida
    The reason is to save operation calls. For example, consider I would first retrieve a directory listing which gives me a sequence<FilePrx>, and then call a method on each FilePrx to get the MD5 values. This would be a lot of overhead. By using a class which holds both the proxy and the MD5 value, I can get a directory listing with all the MD5 values with one call, and then compare the MD5 values with the local values.

    In distributed RPC-based systems, it's quite common to have a "describe" method, which returns a struct or class with data that describes a remote object, instead of having many separate accessor functions. For non-distributed system, no such describe function is necessary, because calls on the object are very cheap. But in distributed systems, each call has a certain overhead, therefore it's better to have one method that returns all the values you need at once.
  • I would recommend to use one single servant for all Ice objects representing files. That's called a default servant.
    [...]
    You could also use an evictor, but in this case, this won't give you much benefit, but it would make your application more complicated.

    Can you please give reasons for your recommendation?
    Is it because files don't have too much behaviours apart from simple request-reply that one servants fulfils all needs?

    What I intend with this question is to know more about the decision proces on when to use

    - default servants
    - non-default servants (through Servant Locators)
    - evictors

    take care,

    Stephan
  • marc
    marc Florida
    The manual discusses the pros and cons in detail. Here is a summary:
    • Regular servants in the Active Servant Map: Simple, easy, ideal for small number of Ice objects.
    • Default servants: Ideal for large number of Ice objects, where the state for Ice objects is not kept in the servants, but instead in a database or in the file system.
    • Evictor: Ideal for large number of Ice objects, where the state for Ice objects is not kept in the servants, but instead in a database or in the file system, but where the servant must cache such state for performance reasons.
  • design question

    Sorry for posting again and again :)

    I took another look at IcePatch.ice and wonder whether this idea makes sense:

    The current interface looks like (simplified):


    interface Directory
    {
    nonmutating long getTotal(Ice::ByteSeq md5);
    };

    class DirectoryDesc
    {
    Directory* dir;
    };


    I now (thanks for your answers) fully understand that you have the description classes for avoiding unecessary network transmissions (for objects that are not likely to be changed too frequently).

    In an implementation, that results into (from IcePatch/Client.cpp):

    total = topDesc->dir->getTotal(md5);


    What I'm now wondering is whether this slice definition change makes sense:


    interface DirectoryRemote
    {
    nonmutating long getTotal(Ice::ByteSeq md5);
    };

    class Directory
    {
    long getTotal(Ice::ByteSeq md5);
    DirectoryRemote* dir;
    };


    with


    Ice::Long Directory::getTotal(Ice::ByteSeq md5, const Ice::Current& current)
    {
    return dir->getTotal(md5);
    }


    imho, this simplifies access to getTotal() since one doesn't even have to know that the interface (interface DirectoryRemote{};) exists but rather has to deal with a single point of access.

    What I now wonder about is whether there are negative implications of this solution.

    take care,

    Stephan
  • marc
    marc Florida
    It's a matter of taste. The advantage of the local class with operations is that you have an abstraction. The disadvantage is that you must install a factory for classes with operations.