From dev-return-82394-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Tue Aug 13 16:38:52 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id D4A661804BB for ; Tue, 13 Aug 2019 18:38:51 +0200 (CEST) Received: (qmail 75072 invoked by uid 500); 13 Aug 2019 16:38:51 -0000 Mailing-List: contact dev-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zookeeper.apache.org Delivered-To: mailing list dev@zookeeper.apache.org Received: (qmail 75061 invoked by uid 99); 13 Aug 2019 16:38:51 -0000 Received: from Unknown (HELO mailrelay1-lw-us.apache.org) (10.10.3.159) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Aug 2019 16:38:51 +0000 Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id B311C84E5 for ; Tue, 13 Aug 2019 16:38:50 +0000 (UTC) Received: by mail-wr1-f44.google.com with SMTP id n9so108476613wru.0 for ; Tue, 13 Aug 2019 09:38:50 -0700 (PDT) X-Gm-Message-State: APjAAAW1Lbqqiw5zCWnkDY+s0TFQdAAJxpoASY+RthqDUXIlCIHfg8OH XXlm93N18gPBvJ/QEZjiEUlN3wS8TtUIW0bNOtE= X-Google-Smtp-Source: APXvYqzFfcZZwwdlcuOoUI8by0ajm3ktN5B7UbHMnYFWJ0IO59+EViEalJJCfti4YBzRNDvod4J8g3/PUVgXsSz956Q= X-Received: by 2002:adf:f6d2:: with SMTP id y18mr46960223wrp.102.1565714330054; Tue, 13 Aug 2019 09:38:50 -0700 (PDT) MIME-Version: 1.0 References: <260A7BE5-42FE-4CA3-9D96-2E5CD32BE159@apache.org> <3071E01E-553E-4FA9-8616-EED8DE0781B5@jordanzimmerman.com> In-Reply-To: From: Patrick Hunt Date: Tue, 13 Aug 2019 09:38:13 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum To: DevZooKeeper Content-Type: multipart/alternative; boundary="0000000000007679080590024634" --0000000000007679080590024634 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Aug 13, 2019 at 9:10 AM Ted Dunning wrote: > If you want to see all events, use Kafka. > > heh - I was thinking the same thing. ;-) Patrick > > > On Tue, Aug 13, 2019 at 8:20 AM Jordan Zimmerman < > jordan@jordanzimmerman.com> > wrote: > > > Also see https://issues.apache.org/jira/browse/ZOOKEEPER-1416 < > > https://issues.apache.org/jira/browse/ZOOKEEPER-1416> > > > > There are many use cases where a client wants to see all events from a > > given parent path down. The semantics of setting one-time watches on a > > single node in ZK are cumbersome for these use cases. FWIW I had a > working > > PR a few years ago but it's fallen far behind 3.6 now. > > > > -Jordan > > > > > On Aug 13, 2019, at 8:18 AM, Andor Molnar wrote: > > > > > > Subscriber API > > > https://issues.apache.org/jira/browse/ZOOKEEPER-153 > > > > > > Is it supposed to be something like a generic Observer API on the > client > > side? > > > Observers essentially consume ordered updates of ZAB, so we would nee= d > > to provide a way for users to implement their own =E2=80=9Cobservers=E2= =80=9D. They > should > > be able to filter for path to be more convenient. > > > > > > Andor > > > > > > > > > > > >> On 2019. Aug 2., at 20:48, Patrick Hunt wrote: > > >> > > >> Michael I think you are describing subscribe - this? > > >> https://issues.apache.org/jira/browse/ZOOKEEPER-153 > > >> wasn't there some work done to keep tlogs around for a while? Or am = I > > miss > > >> remembering? (fb folks?) > > >> > > >> I'll also add that we haven't done any benchmarking in quite some > time. > > It > > >> would be interesting to collect a few of these use cases from the > > >> community, esp downstreams, and evaluate performance, see if we can > > address. > > >> > > >> Patrick > > >> > > >> On Fri, Aug 2, 2019 at 11:03 AM Michael Han wrote: > > >> > > >>> Folks, > > >>> > > >>> Some of you might already see this. Comments? > > >>> > > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooK= eeper+with+a+Self-Managed+Metadata+Quorum > > >>> > > >>> > > >>> What caught my eyes are: > > >>> > > >>> *Worse still, although ZooKeeper is the store of record, the state = in > > >>> ZooKeeper often doesn't match the state that is held in memory in t= he > > >>> controller. For example, when a partition leader changes its ISR i= n > > ZK, > > >>> the controller will typically not learn about these changes for man= y > > >>> seconds. There is no generic way for the controller to follow the > > >>> ZooKeeper event log. Although the controller can set one-shot > > watches, the > > >>> number of watches is limited for performance reasons. When a watch > > >>> triggers, it doesn't tell the controller the current state-- only > that > > the > > >>> state has changed. By the time the controller re-reads the znode a= nd > > sets > > >>> up a new watch, the state may have changed from what it was when th= e > > watch > > >>> originally fired. If there is no watch set, the controller may not > > learn > > >>> about the change at all. In some cases, restarting the controller = is > > the > > >>> only way to resolve the discrepancy.* > > >>> > > >>> I've seen some similar zookeeper use cases that ended up like what'= s > > >>> described here. How can ZooKeeper solve this? It seems to me that t= he > > only > > >>> solution is to provide linearizable read on watched operations. > > Thoughts? > > >>> > > >>> Michael. > > >>> > > > > > > > > --0000000000007679080590024634--