geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jules Gosnell <ju...@coredevelopers.net>
Subject Re: Clustering (long)
Date Fri, 01 Jul 2005 01:52:23 GMT
Jeff Genender wrote:

> Jules,
>
> This is awesome stuff.  I look forward to playing with this stuff in 
> Geronimo.  Let me know if you need a hand with something...


Thanks for the encouragement, Jeff. It was good to meet you at J1. I'll 
give you a shout when things get a little more organised.


Jules

>
> Jeff
>
> Jules Gosnell wrote:
>
>> Guys,
>>
>> I thought that it was high time that I brought you up to date with my
>> efforts in building a clustering layer for Geronimo.
>>
>> The project, wadi.codehaus.org, started as an effort to build a
>> scalable clustered HttpSession implementation, but in doing this, I
>> have built components that should be useful in clustering the state
>> held in any tier of Geronimo e.g. OpenEJB SFSBs etc.
>>
>> WADI (Web Application Distribution Infrastructure) has two main
>> elements - the vertical and the horizontal.
>>
>> Vertically, WADI comprises a stack of pluggable stores. Each store has
>> a pluggable Evicter responsible for demoting aging Sessions
>> downwards. Requests arriving at the container are fed into the top of
>> the stack and progress downwards, until their corresponding Session is
>> found and promoted to the top, where the request is correctly rendered
>> in its presence.
>>
>> Typically the top-level store is in Memory. Aging Sessions are demoted
>> downwards onto exclusively owned LocalDisc. The bottom-most store is a
>> database shared between all nodes in the Cluster. The first node
>> joining the Cluster promotes all Sessions from the database into
>> exclusively-owned store - e.g. LocalDisc. The last node to leave the
>> Cluster demotes all Sessions down back into the database.
>>
>> Horizontally, all nodes in a WADI Cluster are connected (p2p) via a
>> Clustered Store component within this stack. This typically sits at
>> the boundary between exclusive and shared Stores. As requests fall
>> through the stack, looking for their corresponding Session they arrive
>> at the Clustered store, where, if the Session is present anywhere in
>> the Cluster, its location may be learnt. At this point, the Session
>> may be migrated in, underneath the incoming request, or, if its
>> current location is considered advantageous, the request may be
>> proxied or redirected to its remote location. As a node leaves the
>> Cluster, all its Sessions are evacuated to other nodes via this store,
>> so that they may continue to be actively maintained.
>>
>> The space in which Session ids are allocated is divided into a fixed
>> number of Buckets. This number should be large enough such that
>> management of the Buckets may be divided between all nodes in the
>> Cluster roughly evenly. As nodes leave and join the Cluster, a single
>> node, the Coordinator, is responsible for re-Bucketing the Cluster -
>> i.e. reorganising who manages which Buckets and ensuring the safe
>> transfer of the minimum number of Buckets to implement the new
>> layout. The Coordinator is elected via a Pluggable policy. If the
>> Coordinator leaves or fails, a new one is elected. If a node leaves or
>> joins, buckets emigrate from it or immigrate into it, under the
>> control of the Coordinator, to/from the rest of the Cluster.
>>
>> A Session may be efficiently mapped to a Bucket by simply %-ing its
>> ID's hashcode() by the number of Buckets in the Cluster.
>>
>> A Bucket is a map of SessionID:Location, kept up to date with the
>> Location of every Session in the Cluster, of which the id falls into
>> its space. i.e. as Sessions are created, destroyed or moved around the
>> Cluster notifications are sent to the node managing the relevant
>> Bucket, informing it of the change.
>>
>> In this way, if a node receives a request for a Session which it does
>> not own locally, it may pass a message to it, in a maximum of
>> typically two hops, by sending the message to the Bucket owner, who
>> then does a local lookup of the Sessions actual location and forwards
>> the message to it. If Session and Bucket can be colocated, this can
>> reduced to a single hop.
>>
>> Thus, WADI provides a fixed and scalable substrate over the more fluid
>> arrangement that Cluster membership comprises, on top of which further
>> Clustered services may be built.
>>
>> The above functionality exists in WADI CVS and I am currently working
>> on hardening it to the point that I would consider it production
>> strength. I will then consider the addition of some form of state
>> replication, so that, even with the catastrophic failure of a member
>> node, no state is lost from the Cluster.
>>
>> I plan to begin integrating WADI with Geronimo as soon as a certified
>> 1.0-based release starts to settle down. Certification is the most
>> immediate goal and clustering is not really part of the spec, so I
>> think it best to satisfy the first before starting on subsequent
>> goals.
>>
>> Further down the road we need to consider the unification of session
>> id spaces used by e.g. the web and ejb tiers and introduction of an
>> ApplicationSession abstraction - an object which encapsulates all
>> e.g. web and ejb sessions associated with a particular client talking
>> to a particular application. This will allow WADI to maintain the
>> colocation of associated state, whilst moving and replicating it
>> around the Cluster.
>>
>> If anyone would like to know more about WADI, please feel free to ask
>> me questions here on geronimo-dev or on wadi-dev.
>>
>> Thanks for listening,
>>
>>
>> Jules
>>


Mime
View raw message