hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Where are we in ZOOKEEPER-1416
Date Sat, 18 Jan 2014 04:06:50 GMT
On Fri, Jan 17, 2014 at 2:53 PM, Andrew Purtell <apurtell@apache.org> wrote:

> What is going on with this thread over on dev@zookeeper? Bringing it to
> the
> attention of people over here.
>
>
>

Ted?
St.Ack




> ---------- Forwarded message ----------
> From: Ted Dunning <ted.dunning@gmail.com>
> Date: Fri, Jan 17, 2014 at 2:41 PM
> Subject: Re: Where are we in ZOOKEEPER-1416
> To: "dev@zookeeper.apache.org" <dev@zookeeper.apache.org>
>
>
> My reference here is to the comments a ways up thread.  Kishore and I
> clearly agree completely that idempotency and dealing with the state as it
> is right now are the keys to correct design.
>
>
> On Fri, Jan 17, 2014 at 2:14 PM, Ted Dunning <ted.dunning@gmail.com>
> wrote:
>
> >
> > That comment indicates a lack of understanding of ZK, not a bug in ZK.
> >
> > You don't lose state transitions if you read new state at the same time
> > you set the new watch.
> >
> > Likewise, it is simply a product of bad design to have a problem with
> > asynchronous notification.  Changes on other machines *are* asynchronous
> so
> > anybody who can't handle that is inherently denying reality.  If you want
> > to inject the notifications into a sequential view of an event stream,
> that
> > is trivial to do.
> >
> > Systems that depend on transition notification are generally not as
> robust
> > as systems that depend on current state.  Building a cluster manager
> works
> > better if the master is notified that a change has happened, but then
> > simply deals with the situation as it stands.
> >
> > As an analog, imagine that you have a system that shows a number x and a
> > second system that is supposed to show an echo of that number.
> >
> > Design A is notified of changes to x in the form of deltas.  If there is
> > ever an error in handling events, the echo will be off forever.  The
> error
> > that causes the delta to be dropped could be notification or a coding
> error
> > or a misunderstanding of how parallel systems work.  For instance, the
> > InterruptedException might not be handled right.
> >
> > Design B is notified of changes to x and whenever a change happens, the
> > second system simply goes and reads the new state.  Errors will be
> quickly
> > corrected.
> >
> > It sounds like the original poster is trying to build something like
> > Design A when they should be building Design B.
> >
> >
> >
> >
> >
> > On Fri, Jan 17, 2014 at 12:34 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> >> HBASE-5487 is also related.
> >>
> >> The discussion there is very long. Below is an excerpt from Honghua:
> >>
> >> too many tricky scenarios/bugs due to ZK watch is one-time(which can
> >> result
> >> in missed state transition) and the notification/process is
> >> asyncronous(which can lead to delayed/non-update-to-date state in master
> >> memory).
> >>
> >> Cheers
> >>
> >>
> >> On Fri, Jan 17, 2014 at 11:25 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> >>
> >> > Hi, Flavio:
> >> > HBASE-8365 is one such case.
> >> >
> >> > Let me search around for other related discussion.
> >> >
> >> >
> >> > On Fri, Jan 17, 2014 at 11:17 AM, Flavio Junqueira <
> >> fpjunqueira@yahoo.com>wrote:
> >> >
> >> >> Hi Ted,
> >> >>
> >> >> Can you provide more detail on how the precise deltas could make it
> >> more
> >> >> robust?
> >> >>
> >> >> -Flavio
> >> >>
> >> >> -----Original Message-----
> >> >> From: "Ted Yu" <yuzhihong@gmail.com>
> >> >> Sent: 17/01/2014 17:25
> >> >> To: "dev@zookeeper.apache.org" <dev@zookeeper.apache.org>
> >> >> Subject: Re: Where are we in ZOOKEEPER-1416
> >> >>
> >> >> Having the ability to know exact deltas would help make HBase region
> >> >> assignment more robust.
> >> >>
> >> >> Cheers
> >> >>
> >> >>
> >> >>
> >> >> On Fri, Jan 17, 2014 at 9:13 AM, kishore g <g.kishore@gmail.com>
> >> wrote:
> >> >>
> >> >> > I agree with you, I like the side effect and in fact I would prefer
> >> to
> >> >> have
> >> >> > one notification for all changes under a parent node.
> >> >> >
> >> >> > However, Hao is probably asking for ability to know exact deltas.
> >> >> >
> >> >> >
> >> >> > On Fri, Jan 17, 2014 at 8:15 AM, FPJ <fpjunqueira@yahoo.com>
> wrote:
> >> >> >
> >> >> > > We don't need to have a mapping between every change and
a
> >> >> notification.
> >> >> > If
> >> >> > > there are 2+ changes between notifications, you'll be able
to
> >> observe
> >> >> it
> >> >> > by
> >> >> > > reading the ZK state. In fact, one nice side-effect is that
we
> >> reduce
> >> >> the
> >> >> > > number of notifications when there are many concurrent changes.
> >> >> > >
> >> >> > > The only situation I can see it being necessary is the one
in
> >> which we
> >> >> > need
> >> >> > > to know precisely the changes and we haven't cached a previous
> >> >> version of
> >> >> > > the state.
> >> >> > >
> >> >> > > -Flavio
> >> >> > >
> >> >> > > > -----Original Message-----
> >> >> > > > From: kishore g [mailto:g.kishore@gmail.com]
> >> >> > > > Sent: 17 January 2014 16:06
> >> >> > > > To: dev@zookeeper.apache.org
> >> >> > > > Subject: Re: Where are we in ZOOKEEPER-1416
> >> >> > > >
> >> >> > > > I think Hao is pointing out that there is no way to
see every
> >> change
> >> >> > > > (delta) that happened to a znode. Consider 2 changes
A,B in
> quick
> >> >> > > > succession. When client gets notified of A and before
setting
> the
> >> >> watch
> >> >> > > the
> >> >> > > > change B has occurred on the server side. This means
the client
> >> >> cannot
> >> >> > > know
> >> >> > > > the delta A. Client can only read the state after change
B is
> >> >> applied.
> >> >> > > >
> >> >> > > > Implementing the concept of Persistent watcher guarantees
that
> >> >> client
> >> >> > if
> >> >> > > > notified after every change.
> >> >> > > >
> >> >> > > > This is a nice to have feature but I dont understand
the
> >> >> requirement in
> >> >> > > Hbase
> >> >> > > > where this is needed. Hao, can you shed more light on
how this
> >> >> would be
> >> >> > > > useful for HBase (to act like state machine)
> >> >> > > >
> >> >> > > > thanks,
> >> >> > > > Kishore G
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > On Fri, Jan 17, 2014 at 5:18 AM, FPJ <fpjunqueira@yahoo.com>
> >> wrote:
> >> >> > > >
> >> >> > > > > But you don't really miss events, you'll see them
when you
> read
> >> >> the
> >> >> > ZK
> >> >> > > > > state. If you follow the pattern I described, you're
supposed
> >> to
> >> >> > > > > observe all changes. Perhaps I'm missing some concrete
use
> case
> >> >> you
> >> >> > > > > have mind.
> >> >> > > > >
> >> >> > > > > -Flavio
> >> >> > > > >
> >> >> > > > > > -----Original Message-----
> >> >> > > > > > From: 陈迪豪 [mailto:chendihao@xiaomi.com]
> >> >> > > > > > Sent: 17 January 2014 13:03
> >> >> > > > > > To: dev@zookeeper.apache.org
> >> >> > > > > > Subject: RE: Where are we in ZOOKEEPER-1416
> >> >> > > > > >
> >> >> > > > > > No, it's not complicated. But for the people
who don't
> >> >> understand
> >> >> > zk
> >> >> > > > > deeply,
> >> >> > > > > > they would easily ignore the fact that they
would miss
> >> events in
> >> >> > > > > > some
> >> >> > > > > way.
> >> >> > > > > > Moreover, I think providing persistent watch
is good for
> >> >> developers
> >> >> > > > > > to
> >> >> > > > > build
> >> >> > > > > > the "state-machine" application. Actually,
HBase suffer
> from
> >> >> > missing
> >> >> > > > > > the intermediate state when use zk to store
the data.
> >> >> > > > > >
> >> >> > > > > > If the feature is implemented, I would like
to see the
> patch
> >> and
> >> >> > > > > > consider
> >> >> > > > > if it
> >> >> > > > > > can be used for us.
> >> >> > > > > >
> >> >> > > > > > ________________________________________
> >> >> > > > > > From: Flavio Junqueira [fpjunqueira@yahoo.com]
> >> >> > > > > > Sent: Friday, January 17, 2014 8:12 PM
> >> >> > > > > > To: dev@zookeeper.apache.org
> >> >> > > > > > Subject: RE: Where are we in ZOOKEEPER-1416
> >> >> > > > > >
> >> >> > > > > > My take is that persistent subscriptions add
complexity and
> >> are
> >> >> not
> >> >> > > > > strictly
> >> >> > > > > > necessary. You can follow this pattern of
setting a watch,
> >> >> reading
> >> >> > > > > > the
> >> >> > > > > state
> >> >> > > > > > upon a notification and setting a new watch.
Why do you
> feel
> >> >> that's
> >> >> > > > > > complicated?
> >> >> > > > > >
> >> >> > > > > > -Flavio
> >> >> > > > > >
> >> >> > > > > > -----Original Message-----
> >> >> > > > > > From: 陈迪豪 [mailto:chendihao@xiaomi.com]
> >> >> > > > > > Sent: Friday, January 17, 2014 3:13 AM
> >> >> > > > > > To: dev@zookeeper.apache.org
> >> >> > > > > > Subject: Where are we in ZOOKEEPER-1416
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > Persistent watch and implementing the feature
to act like
> >> "state
> >> >> > > > machine"
> >> >> > > > > > which is mentioned in
> >> >> > > > > > ZOOKEEPER-153<
> >> https://issues.apache.org/jira/browse/ZOOKEEPER-
> >> >> > > > 153>,
> >> >> > > > > > would be great for ZooKeeper user. I think
HBase would like
> >> to
> >> >> know
> >> >> > > > > > all
> >> >> > > > > the
> >> >> > > > > > change in zk rather than missing kind of events.
> >> >> > > > > >
> >> >> > > > > > So, would we continue developing these features?
It's also
> a
> >> >> little
> >> >> > > > > > complicated to develop with zk and I think
there're lots of
> >> >> things
> >> >> > > > > > to
> >> >> > > > > improve.
> >> >> > > > >
> >> >> > > > >
> >> >> > > > >
> >> >> > >
> >> >> > >
> >> >> >
> >> >>
> >> >
> >> >
> >>
> >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message