zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andor Molnar <an...@apache.org>
Subject Re: KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
Date Tue, 13 Aug 2019 17:41:04 GMT
Hmmm... and what should Kafka do if it wants to see all events? Use
RabbitMQ? :)
Andor

-----Original Message-----From: Ted Dunning <ted.dunning@gmail.com>Repl
y-To: dev@zookeeper.apache.orgTo: dev@zookeeper.apache.orgSubject: Re:
KIP-500: Replace ZooKeeper with a Self-Managed Metadata QuorumDate:
Tue, 13 Aug 2019 09:10:04 -0700
If you want to see all events, use Kafka.


On Tue, Aug 13, 2019 at 8:20 AM Jordan Zimmerman <
jordan@jordanzimmerman.com>wrote:
> Also see https://issues.apache.org/jira/browse/ZOOKEEPER-1416 <
> https://issues.apache.org/jira/browse/ZOOKEEPER-1416>
> There are many use cases where a client wants to see all events from
> agiven parent path down. The semantics of setting one-time watches on
> asingle node in ZK are cumbersome for these use cases. FWIW I had a
> workingPR a few years ago but it's fallen far behind 3.6 now.
> -Jordan
> > On Aug 13, 2019, at 8:18 AM, Andor Molnar <andor@apache.org> wrote:
> > Subscriber APIhttps://issues.apache.org/jira/browse/ZOOKEEPER-153
> > 
> > Is it supposed to be something like a generic Observer API on the
> > client
> side?
> > Observers essentially consume ordered updates of ZAB, so we would
> > need
> to provide a way for users to implement their own “observers”. They
> shouldbe able to filter for path to be more convenient.
> > Andor
> > 
> > 
> > > On 2019. Aug 2., at 20:48, Patrick Hunt <phunt@apache.org> wrote:
> > > Michael I think you are describing subscribe - this?
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-153
> > > wasn't there some work done to keep tlogs around for a while? Or
> > > am I
> miss
> > > remembering? (fb folks?)
> > > I'll also add that we haven't done any benchmarking in quite some
> > > time.
> It
> > > would be interesting to collect a few of these use cases from
> > > thecommunity, esp downstreams, and evaluate performance, see if
> > > we can
> address.
> > > Patrick
> > > On Fri, Aug 2, 2019 at 11:03 AM Michael Han <hanm@apache.org>
> > > wrote:
> > > > Folks,
> > > > Some of you might already see this. Comments?
> > > > 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum
> > > > What caught my eyes are:
> > > > *Worse still, although ZooKeeper is the store of record, the
> > > > state inZooKeeper often doesn't match the state that is held in
> > > > memory in thecontroller.  For example, when a partition leader
> > > > changes its ISR in
> ZK,
> > > > the controller will typically not learn about these changes for
> > > > manyseconds.  There is no generic way for the controller to
> > > > follow theZooKeeper event log.  Although the controller can set
> > > > one-shot
> watches, the
> > > > number of watches is limited for performance reasons.  When a
> > > > watchtriggers, it doesn't tell the controller the current
> > > > state-- only that
> the
> > > > state has changed.  By the time the controller re-reads the
> > > > znode and
> sets
> > > > up a new watch, the state may have changed from what it was
> > > > when the
> watch
> > > > originally fired.  If there is no watch set, the controller may
> > > > not
> learn
> > > > about the change at all.  In some cases, restarting the
> > > > controller is
> the
> > > > only way to resolve the discrepancy.*
> > > > I've seen some similar zookeeper use cases that ended up like
> > > > what'sdescribed here. How can ZooKeeper solve this? It seems to
> > > > me that the
> only
> > > > solution is to provide linearizable read on watched operations.
> Thoughts?
> > > > Michael.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message