zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
Date Tue, 13 Aug 2019 16:38:13 GMT
On Tue, Aug 13, 2019 at 9:10 AM Ted Dunning <ted.dunning@gmail.com> wrote:

> If you want to see all events, use Kafka.
>
>
heh - I was thinking the same thing. ;-)

Patrick


>
>
> On Tue, Aug 13, 2019 at 8:20 AM Jordan Zimmerman <
> jordan@jordanzimmerman.com>
> wrote:
>
> > Also see https://issues.apache.org/jira/browse/ZOOKEEPER-1416 <
> > https://issues.apache.org/jira/browse/ZOOKEEPER-1416>
> >
> > There are many use cases where a client wants to see all events from a
> > given parent path down. The semantics of setting one-time watches on a
> > single node in ZK are cumbersome for these use cases. FWIW I had a
> working
> > PR a few years ago but it's fallen far behind 3.6 now.
> >
> > -Jordan
> >
> > > On Aug 13, 2019, at 8:18 AM, Andor Molnar <andor@apache.org> wrote:
> > >
> > > Subscriber API
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-153
> > >
> > > Is it supposed to be something like a generic Observer API on the
> client
> > side?
> > > Observers essentially consume ordered updates of ZAB, so we would need
> > to provide a way for users to implement their own “observers”. They
> should
> > be able to filter for path to be more convenient.
> > >
> > > Andor
> > >
> > >
> > >
> > >> On 2019. Aug 2., at 20:48, Patrick Hunt <phunt@apache.org> wrote:
> > >>
> > >> Michael I think you are describing subscribe - this?
> > >> https://issues.apache.org/jira/browse/ZOOKEEPER-153
> > >> wasn't there some work done to keep tlogs around for a while? Or am I
> > miss
> > >> remembering? (fb folks?)
> > >>
> > >> I'll also add that we haven't done any benchmarking in quite some
> time.
> > It
> > >> would be interesting to collect a few of these use cases from the
> > >> community, esp downstreams, and evaluate performance, see if we can
> > address.
> > >>
> > >> Patrick
> > >>
> > >> On Fri, Aug 2, 2019 at 11:03 AM Michael Han <hanm@apache.org> wrote:
> > >>
> > >>> Folks,
> > >>>
> > >>> Some of you might already see this. Comments?
> > >>>
> > >>>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum
> > >>>
> > >>>
> > >>> What caught my eyes are:
> > >>>
> > >>> *Worse still, although ZooKeeper is the store of record, the state
in
> > >>> ZooKeeper often doesn't match the state that is held in memory in the
> > >>> controller.  For example, when a partition leader changes its ISR in
> > ZK,
> > >>> the controller will typically not learn about these changes for many
> > >>> seconds.  There is no generic way for the controller to follow the
> > >>> ZooKeeper event log.  Although the controller can set one-shot
> > watches, the
> > >>> number of watches is limited for performance reasons.  When a watch
> > >>> triggers, it doesn't tell the controller the current state-- only
> that
> > the
> > >>> state has changed.  By the time the controller re-reads the znode and
> > sets
> > >>> up a new watch, the state may have changed from what it was when the
> > watch
> > >>> originally fired.  If there is no watch set, the controller may not
> > learn
> > >>> about the change at all.  In some cases, restarting the controller
is
> > the
> > >>> only way to resolve the discrepancy.*
> > >>>
> > >>> I've seen some similar zookeeper use cases that ended up like what's
> > >>> described here. How can ZooKeeper solve this? It seems to me that the
> > only
> > >>> solution is to provide linearizable read on watched operations.
> > Thoughts?
> > >>>
> > >>> Michael.
> > >>>
> > >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message