geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jules Gosnell <ju...@coredevelopers.net>
Subject Re: [wadi-dev] Re: [Geronimo] Clustering
Date Mon, 19 Dec 2005 18:56:22 GMT
James Strachan wrote:

> On 19 Dec 2005, at 17:25, Jules Gosnell wrote:
>
>> James Strachan wrote:
>>
>>> On 19 Dec 2005, at 14:54, Jules Gosnell wrote:
>>>
>>>> James Strachan wrote:
>>>>
>>>>> On 19 Dec 2005, at 14:14, Jules Gosnell wrote:
>>>>>
>>>>>> James Strachan wrote:
>>>>>>
>>>>>>> On 19 Dec 2005, at 11:53, Jules Gosnell wrote:
>>>>>>>
>>>>>>>> , whether there is other suitable Geronimo or ASF-licensed
  
>>>>>>>> code   available, or whether we will need to write our own
  
>>>>>>>> WADI-  autodiscovery classes. The important thing is to 

>>>>>>>> impose  as few   dependencies on the client as possible.
The  
>>>>>>>> client  side code  should  literally be a few lines. Clients
 
>>>>>>>> using  clusters should  not  suddenly find themselves sucking
 
>>>>>>>> down  e.g. the whole of  activemq,  just to do a once off
  
>>>>>>>> autodiscovery. Early versions of  WADI had its  own   
>>>>>>>> autodiscovery code. If we need them, they could  be  resuscitated.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> There's no reason why you can't do a simple implementation  
>>>>>>> of    ActiveCluster which doesn't use ActiveMQ - its just a 

>>>>>>> simple API.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Sure - but I'm talking about the EJB-client side - where we  
>>>>>> just   want to throw across as thin a line as possible, in  order

>>>>>> to haul  a  decent strength cable back. An EJB client  would not

>>>>>> need the   ActiveCluster API (I'm not thinking in  terms of 
>>>>>> making EJB  clients  fully fledged cluster members),  but simply

>>>>>> a way of  locating the  cluster and requesting a  membership 
>>>>>> snapshot of it.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Thats exactly what the ActiveCluster API is for :). Though by  
>>>>> all   means come up with another API if you can think of a  better 
>>>>> way  of  doing it.
>>>>>
>>>>>> This could be done by just broadcasting a query packet at a  
>>>>>> well   known multicast address and waiting for the first well- 
>>>>>> formed   response.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Sure - an *implementation* of ActiveCluster API could do  exactly  
>>>>> that.
>>>>>
>>>> ???
>>>>
>>>> well, maybe I'm thinking of the wrong piece of activecluster then ?
>>>>
>>>> any piece of code could broadcast a packet... which piece of   
>>>> activecluster's API are you suggesting here ?
>>>
>>>
>>>
>>>
>>> You said...
>>>
>>>
>>>>>> but simply a way of locating the  cluster and requesting a   
>>>>>> membership snapshot of it.
>>>>>
>>>>>
>>>
>>> The entire purpose of the ActiveCluster APi is to locate members  
>>> in  clusters & be informed when they go down. There's a zillion  
>>> ways of  implementing this - AC just tries to be a common, simple  
>>> API we can  share.
>>
>>
>> Sure
>>
>>>
>>>
>>>> we really are talking about just a remoting proxy which needs to   
>>>> find, but not 'join' a cluster.
>>>
>>>
>>>
>>> AC can be used purely to discover nodes and choose which one to  
>>> use  as a server. The local in-JVM node does not actually have to  
>>> actually  advertise itself to members of the cluster if it doesn't  
>>> wish to -  thats an implementation detail. (e.g. if you only want  
>>> servers to be  discoverable but keep clients invisible).
>>
>>
>> OK - I see - but I'm not sure that I am drawn to this approach :-)
>
>
> Why?
>
>
>>> Using the EJB client and server side example - you may want to  
>>> know  which clients are connected to what servers so that you can  
>>> load  balance; so even though you may not consider clients as  
>>> 'members of  the server cluster' being able to know who is  
>>> connected to who is  quite useful. EJB clients could expose  
>>> statistics that can be used to  help choose the right server a new  
>>> node should join.
>>
>>
>> understood.
>>
>>>
>>> But whatever - if you don't wanna use the AC API its no big deal;  
>>> use  whatever abstraction you see fit - I just wanted to explain  
>>> the point  of AC; for discovering nodes in a cluster and being  able 
>>> to choose  which one to communicate with; it can be  implemented any 
>>> way you like  (simple multicast stuff, using HTTP,  LDAP stuff etc).
>>
>>
>> Its not that I don't want to use AC client side - its that I don't  
>> see clients as being members of the cluster in any way - passive or  
>> active.
>>
>> but, I guess this opens up an interesting topic...
>>
>> I'm thinking in terms of relatively large numbers of clients to  
>> servers.
>>
>> I wasn't thinking in terms of clients receiving any notifications  
>> about changes in cluster membership or state location.
>>
>> This is probably because WADI comes from a background in which the  
>> client is completely dumb, but also because a server may support  
>> many thousands of clients. I see the overhead of trying to  
>> pessimistically keep these clients in sync with developments within  
>> the cluster as outweighing the benefit.
>
>
> Who said anything about state synchronization? We're talking  
> discovery here.
>
>
>> The architecture that I have in mind is more optimistic. It waits  
>> for something exceptional to happen - e.g. some state migrating as  
>> its node shuts down followed by a client for that state trying to  
>> make a call to it - then pays a small but exceptional cost in terms  
>> of a fail-over (which may become an autodiscovery in the even more  
>> exceptional case that all nodes in the cluster have been cycled  
>> between successive calls from the client).
>>
>> Thus, in the usecase that I expect, I think that the optimistic  
>> approach will be more efficient than the more pessimistic. However  
>> you should expect the unexpected - right :-)
>>
>> I guess that this way of thinking has prevented me from seeing  
>> clients as cluster members.... when perhaps they might be seen that  
>> way ?
>
>
> As I said, there's no requirement in AC that clients are cluster  
> members. You can use AC purely to observe the *servers* only if that  
> what you want - to discover new servers and detect if they go down.  
> You could have 1M clients just watching the 3 servers and auto- 
> detecting new servers come along and go away.
>
>
>> Perhaps we could look at how AC impls auto discover cluster  members, 
>> then abstract that out into another interface (Client)  which Node 
>> could inherit/make use of ?
>
>
> Huh?
>
> BTW I'll say it one more time, AC == a simple API with pluggable  
> implementations - you can implement it however you want.
>
> AC is just a really trivial API for watching a group of Nodes; a Node  
> can be anything you want it to be Jules - and you can implement the  
> API in any way you choose. So it could be just a heartbeat multicast  
> packet sent by each server now and again (which is pretty much how  
> the current implementation works by default). If you really want a  
> different API then make one. But the number 1 reason for AC's  
> existence was to auto-discover servers and allow clients to connect  
> to them in some way, which seems to be pretty similar to the EJB use  
> case your discussing.
>
I understand exactly what AC is. WADI relies totally upon it, so I come 
into contact with it daily. I love it ! I just don't, as yet, see it in 
the EJB client stub.

I agree that there is overlap in terms of what AC can do and what the 
client needs (in fact AC completely subsumes this fn-ality - as you have 
said), but I think that AC's API (I'm talking about Cluster and Node, 
perhaps you had other parts of the API in mind) is much broader than 
what is required here. In any other situation, I would probably go with 
AC, but because we are talking about a client stub in a third party 
application I am putting minimalism very high on my list.

Let us temporarily agree to disagree. I promise to give this further 
consideration.

Ultimately, I think that it is unlikely that I will write this code, and 
I exhort the person who does to consider James' and my views equally and 
come to their own conclusion. If they would like to talk more about it I 
am sure that we will both still be around to discuss it :-)


Jules

> James
> -------
> http://radio.weblogs.com/0112098/



-- 
"Open Source is a self-assembling organism. You dangle a piece of
string into a super-saturated solution and a whole operating-system
crystallises out around it."

/**********************************
 * Jules Gosnell
 * Partner
 * Core Developers Network (Europe)
 *
 *    www.coredevelopers.net
 *
 * Open Source Training & Support.
 **********************************/


Mime
View raw message