Archived

This forum has been archived. Please start a new discussion on GitHub.

Freeze questions (from ACM paper)

The ACM paper is very interesting, though there are some things that raised questions for me. (Does the paper describe an unreleased version of Ice, perhaps?)

My first questions are about the Freeze persistence system. The Ice book says, in 21.5.2:
A modified Ice object is one on which a mutating operation has been invoked since the last time the object's state was saved.
The ACM Queue paper, though, says:
To update objects, developers simply assign to their state attributes.
Does this mean that an "internal" update to a Freeze-managed object will somehow lead to an update in the persistent storage? (Imagine, perhaps, a thread started by a persistent servant when it's thawed, which makes periodic updates to its state.)

I'm also curious about the relationship between the (righteous!) federation support described for load balancing, and Freeze-persisted servants. If multiple servers are federated behind the randomizing load-balancer, how is the persistent state communicated between different servers? I could see a system based on hashing of the identity, which is what I had in mind for load-balancing my own servers, but maybe there's something more clever afoot? (A shared-storage SAN might be one option, but that sounds like a way to get badly burned by the different page caches in play.)

More later, I suspect!

Mike

Comments

  • marc
    marc Florida
    Re: Freeze questions (from ACM paper)
    Originally posted by shaver
    Does this mean that an "internal" update to a Freeze-managed object will somehow lead to an update in the persistent storage? (Imagine, perhaps, a thread started by a persistent servant when it's thawed, which makes periodic updates to its state.)

    That's correct. You implement methods for an Ice object that update the state by simply assigning the state to the data members. As long as you always use a proxy to call these methods (and not call them directly on the servant), Freeze will make sure that the state is automatically stored in the database, or that the servant for the Ice object is loaded on demand and later evicted.
    Originally posted by shaver
    I'm also curious about the relationship between the (righteous!) federation support described for load balancing, and Freeze-persisted servants. If multiple servers are federated behind the randomizing load-balancer, how is the persistent state communicated between different servers? I could see a system based on hashing of the identity, which is what I had in mind for load-balancing my own servers, but maybe there's something more clever afoot? (A shared-storage SAN might be one option, but that sounds like a way to get badly burned by the different page caches in play.)

    The state is not communicated. Generally speaking, without taking "fantasy implementations" into account, objects with fast changing state on the one hand, and replicated and load-balanced servers on the other hand, are mutually exclusive. The only real exception are objects with slowly changing state, or designs where you have a main server and one or more hot standby servers.
  • Re: Re: Freeze questions (from ACM paper)
    Originally posted by marc
    The state is not communicated. Generally speaking, without taking "fantasy implementations" into account, objects with fast changing state on the one hand, and replicated and load-balanced servers on the other hand, are mutually exclusive. The only real exception are objects with slowly changing state, or designs where you have a main server and one or more hot standby servers.
    OK, thanks. So for what sorts of objects does Wish use this randomized load balancing? Is it just "stateless" services like pathfinding? I (mis?)understood the description in the "Scalability" section of the paper as describing a more general object-replicating solution, but as you point out, that's a very hard nut to crack. I was very interested in the mention of doing "hot" server upgrades, but I guess that could rely on the "send a class" object migration described.

    Mike
  • matthew
    matthew NL, Canada
    Re: Re: Re: Freeze questions (from ACM paper)
    Originally posted by shaver
    OK, thanks. So for what sorts of objects does Wish use this randomized load balancing? Is it just "stateless" services like pathfinding? I (mis?)understood the description in the "Scalability" section of the paper as describing a more general object-replicating solution, but as you point out, that's a very hard nut to crack. I was very interested in the mention of doing "hot" server upgrades, but I guess that could rely on the "send a class" object migration described.

    Mike

    Yes, Wish mainly uses randomized load balancing for the pathfinding. Wish can also do hot server upgrades of many of the internal services. However, existing objects are not currently migrated to these servers -- they are only utilitized for new objects. However, since the world is constantly changing and objects are being created and destroyed on a continual basis this has not turned out to be an issue in practice.

    Regards, Matthew
  • Re: Re: Re: Re: Freeze questions (from ACM paper)
    Originally posted by matthew
    Yes, Wish mainly uses randomized load balancing for the pathfinding. Wish can also do hot server upgrades of many of the internal services. However, existing objects are not currently migrated to these servers -- they are only utilitized for new objects. However, since the world is constantly changing and objects are being created and destroyed on a continual basis this has not turned out to be an issue in practice.
    That's very interesting, thank you. I assume that the "Character" servers aren't then subject to this sort of hot upgrade until all clients have logged out and then back in. I had assumed that things like "spawncontrollers" (not individual mobs; maybe wish doesn't have that sort of architecture) and "props" and NPCs would have very long lifetimes, but maybe those edge cases aren't a big deal.

    Out of curiosity, what does wish use the migrate-objects-by-sending-Ice-class technique for, then? I've been trying to find a good design case for class-with-behaviour for object "sending". Currently, I only use classes for Freeze implementations, really.

    Mike
  • matthew
    matthew NL, Canada
    Re: Re: Re: Re: Re: Freeze questions (from ACM paper)
    Originally posted by shaver
    That's very interesting, thank you. I assume that the "Character" servers aren't then subject to this sort of hot upgrade until all clients have logged out and then back in. I had assumed that things like "spawncontrollers" (not individual mobs; maybe wish doesn't have that sort of architecture) and "props" and NPCs would have very long lifetimes, but maybe those edge cases aren't a big deal.

    The characters are subject to hot upgrade, but as I said it only applies to a new character. The old characters currently remain on the current server. Its possible to migrate them manually, but we haven't bothered to this point.

    The spawncontrollers reside with a mob itself, or a prop (like a beehive) and so do not migrate. NPCs do have a longer lifespan, but they can be re-created if it becomes necessary.

    Out of curiosity, what does wish use the migrate-objects-by-sending-Ice-class technique for, then? I've been trying to find a good design case for class-with-behaviour for object "sending". Currently, I only use classes for Freeze implementations, really.

    Mike

    Yes, this occurs, for example, when you pick up an object. The object is migrated to the server upon which the character is present.

    Matthew