Mailing-List: contact zookeeper-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: zookeeper-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Subject: RE: Leader Elections
Date: Mon, 20 Jul 2009 15:04:01 -0700
Message-ID: 
 <755F6B792C85484E8665C01CE5E0F74101709A02@corpdc-exch01.corp.digimine.com>
In-Reply-To: <b54edc3a0907201453i1a0ecd07he8948ce8f9536ae8@mail.gmail.com>
Thread-Topic: Leader Elections
Thread-Index: AcoJhPc2r0Az1W0AReyqDct0xGs/NgAAHxnQ
References: <b54edc3a0907191216k1f889ecbo40c996fd46532ec5@mail.gmail.com>
 <C689FEC3.A7F3%scott@richrelevance.com>
 <755F6B792C85484E8665C01CE5E0F7410170999C@corpdc-exch01.corp.digimine.com>
 <b54edc3a0907201453i1a0ecd07he8948ce8f9536ae8@mail.gmail.com>
From: "Todd Greenwood" <toddg@audiencescience.com>
To: <zookeeper-user@hadoop.apache.org>

Henry, cool. When youre patch is ready for testing, I'll devote some
time to take a test pass on it.

-----Original Message-----
From: Henry Robinson [mailto:henry@cloudera.com]=20
Sent: Monday, July 20, 2009 2:54 PM
To: zookeeper-user@hadoop.apache.org
Subject: Re: Leader Elections

On Mon, Jul 20, 2009 at 7:50 PM, Todd Greenwood
<toddg@audiencescience.com>wrote:

> Flavio, Ted, Henry, Scott, this would perfectly well for my use case
> provided:
>
> SINGLE ENSEMBLE:
>        GROUP A : ZK Servers w/ read/write AND Leader Elections
>        GROUP B : ZK Servers w/ read/write W/O Leader Elections
>
> So, we can craft this via Observers and Hiererarchial Quorum groups?
> Great. Problem solved.
>
> When will this be production ready? :o)
>

Looks to me like you don't even need hierarchical quorums for this -
make
everyone in group B an Observer and you're done.

I've been working on this feature. Recently we've been discussing a
proof-of-concept patch on the JIRA. I have nearly finished a less rough
patch which I will submit for discussion and potentially commit this
week.
At that point it would be extremely helpful if you could help test the
patch, and you can start considering it for production. To get into
trunk I
will have to write a comprehensive test suite and update the
documentation,
and then making sure all the boxes are ticked and no regressions are
thrown
up can take a little while.

Henry


>
> --------------------
>
> Scott brought up a multi-feature that is very interesting for me.
> Namely:
>
> 1. Offline ZK servers that sync & merge on reconnect
>
> The offline servers seems conceptually simple, it's kind of like a
> messaging system. However, the merge and resolve step when two servers
> reconnect might be challenging. Cool idea though.
>
> 2. Partial memory graph subscriptions
>
> The second idea is partial memory graph subscriptions. This would
enable
> virtual ensembles to interract on the same physical ensemble. For my
use
> case, this would prevent unnecessary cross talk between nodes on a
WAN,
> allowing me to define the subsets of the memory graph that need to be
> replicated, and to whom. This would be a huge scalability win for WAN
> use cases.
>
> -Todd
>
> -----Original Message-----
> From: Scott Carey [mailto:scott@richrelevance.com]
> Sent: Monday, July 20, 2009 11:00 AM
> To: zookeeper-user@hadoop.apache.org
> Subject: Re: Leader Elections
>
> Observers would be awesome especially with a couple enhancements /
> extensions:
>
> An option for the observers to enter a special state if the WAN link
> goes down to the "master" cluster.  A read-only option would be great.
> However, allowing certain types of writes to continue on a limited
basis
> would be highly valuable as well.  An observer could "own" a special
> node and its subnodes.  Only these subnodes would be writable by the
> observer when there was a session break to the master cluster, and the
> master cluster would take all the changes when the link is
> reestablished.  Essentially, it is a portion of the hierarchy that is
> writable only by a specitfic observer, and read-only for others.
> The purpose of this would be for when the WAN link goes down to the
> "master" ZKs for certain types of use cases - status updates or other
> changes local to the observer that are strictly read-only outside the
> Observer's 'realm'.
>
>
> On 7/19/09 12:16 PM, "Henry Robinson" <henry@cloudera.com> wrote:
>
> You can. See ZOOKEEPER-368 - at first glance it sounds like observers
> will
> be a good fit for your requirements.
>
> Do bear in mind that the patch on the jira is only for discussion
> purposes;
> I would not consider it currently fit for production use. I hope to
put
> up a
> much better patch this week.
>
> Henry
>
> On Sat, Jul 18, 2009 at 7:38 PM, Ted Dunning <ted.dunning@gmail.com>
> wrote:
>
> > Can you submit updates via an observer?
> >
> > On Sat, Jul 18, 2009 at 6:38 AM, Flavio Junqueira
<fpj@yahoo-inc.com>
> > wrote:
> >
> > > 2- Observers: you could have one computing center containing an
> ensemble
> > > and observers around the edge just learning committed values.
> >
> >
> >
> >
> > --
> > Ted Dunning, CTO
> > DeepDyve
> >
>
>