zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Reed <br...@yahoo-inc.com>
Subject Re: Zookeeper WAN Configuration
Date Sat, 25 Jul 2009 06:48:19 GMT
the processing of the write transaction is described in the zookeeper 
internals presentation on 
http://wiki.apache.org/hadoop/ZooKeeper/ZooKeeperPresentations i think 
other presentations may also touch on it. we also have it in the 
ZooKeeper documentation: 
http://hadoop.apache.org/zookeeper/docs/r3.2.0/zookeeperInternals.html

ben


Todd Greenwood wrote:
> Flavio & Ted, thank you for your comments.
>
> So it sounds like the only way to currently deploy to the WAN is to
> deploy ZK Servers to the central DC and open up client connections to
> these ZK servers from the edge nodes. True?
>
> In the future, once the Observers feature is implemented, then we should
> be able to deploy zk servers to both the DC and to the pods...with all
> the goodness that Flavio mentions below.
>
> Flavio - do you have a doc that describes exactly what happens in the
> transaction of a write operation? For instance, I'd like to know at
> exactly what stage a write has been commited to the ensemble, and not
> just the zk server the client is connected to. I figure it must be
> something like:
>
> clientA.write(path, value)
> -> serverA writes to memory
> -> serverA writes to transacted disk every n/seconds or m/bytes
> -> serverA sends write to Leader
> -> Leader stamps with transaction id
> -> Leader responds to ensemble with update + transaction id
>
> -Todd
>
> -----Original Message-----
> From: Flavio Junqueira [mailto:fpj@yahoo-inc.com] 
> Sent: Friday, July 24, 2009 4:50 PM
> To: zookeeper-user@hadoop.apache.org
> Subject: Re: Zookeeper WAN Configuration
>
> Just a few quick observations:
>
> On Jul 24, 2009, at 4:40 PM, Ted Dunning wrote:
>
>   
>> On Fri, Jul 24, 2009 at 4:23 PM, Todd Greenwood
>> <toddg@audiencescience.com>wrote:
>>
>>     
>>> Could you explain the idea behind the Observers feature, what this
>>> concept is supposed to address, and how it applies to the WAN
>>> configuration problem in particular?
>>>
>>>       
>> Not really.  I am just echoing comments on observers from them that  
>> know.
>>
>>     
>
> Without observers, increasing the number of servers in an ensemble  
> enables higher read throughput, but causes write throughput to drop  
> because the number of votes to order each write operation increases.  
> Essentially, observers are zookeeper servers that don't vote when  
> ordering updates to the zookeeper state. Adding observers enables  
> higher read throughput affecting minimally write throughput (leader  
> still has to send commits to everyone, at least in the version we have  
> been working on).
>
>   
>>> """
>>> The ideas for federating ZK or allowing observers would likely do  
>>> what
>>> you
>>> want.  I can imagine that an observer would only care that it can see
>>> it's
>>> local peers and one of the observers would be elected to get updates
>>> (and
>>> thus would care about the central service).
>>> """
>>> This certainly sounds like exactly what I want...Was this  
>>> introduced in
>>> 3.2 in full, or only partially?
>>>
>>>       
>> I don't think it is even in trunk yet.  Look on Jira or at the  
>> recent logs
>> of this mailing list.
>>     
>
> It is not on trunk yet.
>
> -Flavio
>
>   


Mime
View raw message