From user-return-11220-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Wed Jan 24 11:15:58 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 97A19180630 for ; Wed, 24 Jan 2018 11:15:58 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 887AA160C2E; Wed, 24 Jan 2018 10:15:58 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id A7386160C3C for ; Wed, 24 Jan 2018 11:15:57 +0100 (CET) Received: (qmail 33396 invoked by uid 500); 24 Jan 2018 10:15:56 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 33089 invoked by uid 99); 24 Jan 2018 10:15:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Jan 2018 10:15:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 2F35D180416 for ; Wed, 24 Jan 2018 10:15:55 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=cloudera.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id jgOUK6Ewlomz for ; Wed, 24 Jan 2018 10:15:53 +0000 (UTC) Received: from mail-oi0-f54.google.com (mail-oi0-f54.google.com [209.85.218.54]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 1EABD5F590 for ; Wed, 24 Jan 2018 10:15:53 +0000 (UTC) Received: by mail-oi0-f54.google.com with SMTP id d124so2467497oib.13 for ; Wed, 24 Jan 2018 02:15:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudera.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=ab9bI1PQkWrEQYiZp4phWPL04YiA8lCOTKgEmlpMU7Q=; b=PW/5jO5b5VaCiWf1aRenwjnm0AdnGnVfqQw+F9YgaYEBqmn3YLBWRP5BOQDPwpWnTq z7Iu+PKnuDGPqcBthpEmzz9Rv5HKOs5j5MXK04e546Iw2TfHbE33M3PxNdbH/1HUhV2d mJqMAdirfYg3j6BBrAtZHfdIYvWDWXBr/GmmAGF7NDVi8QGzTZb6FrB/krlRrvB9Y8Rq u+6eAD20klE3eXVQtV8JCAfrla7pz5QtsPwW/NBtYmT1VbTfXF2JQVdo0JnzRKc8yT/E e8sxgqJb5tHhsp2PzI9/z+1bmroSC0YCShfEPeTJHqe4mZq5UA5t5Pr4RfiHSwct2PZD jrdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=ab9bI1PQkWrEQYiZp4phWPL04YiA8lCOTKgEmlpMU7Q=; b=NSiCodF4yMgejApPzoDggBmwQyUgAhBlQUFpdJ1TSaofAz2OSPFy1lpMz5CZZbASLI seWsHIDwxZpB03fERD8liYBrONBH7MpPwQpyUjHbFtgdLxwuKu/SXVI5e+CrXLCBxzmp ljG70Gpvd5a07BM4cq0lAPJETJ8XGh9XxVfRdpiC5CqIQZJTadhCfgog706OJePerfy7 e+7GvjiATox8vfphZAiVBOx/H/zfBUgzrbZb8anfhGIZx0KRARbAgrvOHZLnGWC1lJt3 UBHrHtlN7jUo3GIa8qCue2xKvkA04YYa3ICu6d7i9iXmsDR12ykE3mp5AApKOXIJy5LG BvhA== X-Gm-Message-State: AKwxytdgIYRBT6NiQfvQF+sKnwDSdAgbNKsQGuOSo7KmMD/s+hrTHjhp yyPuOnZRLp1Gov9ihRGQl0VX5ptdYkQSosiBdU/QGzzR X-Google-Smtp-Source: AH8x224naWpbJ1QL+Wu5EoWHgi3vERsG17tXiQ2Dcjh9AkwR6WYR21hBuv8TiC/+qURPEXUuocmBNuVOuHZPHp51v5U= X-Received: by 10.202.9.11 with SMTP id 11mr634613oij.105.1516788950885; Wed, 24 Jan 2018 02:15:50 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.91.23 with HTTP; Wed, 24 Jan 2018 02:15:50 -0800 (PST) In-Reply-To: References: From: Andor Molnar Date: Wed, 24 Jan 2018 11:15:50 +0100 Message-ID: Subject: Re: Observers: a link to message bus? To: user@zookeeper.apache.org Content-Type: multipart/alternative; boundary="f4f5e808e28c9e11d1056382f2d3" --f4f5e808e28c9e11d1056382f2d3 Content-Type: text/plain; charset="UTF-8" Hi Bruno, Unfortunately I have no idea what plugin system does the author of this doc refer to, but I can think of the following 2 solutions: 1) Fire up some observers in ZK cluster and connect to them via some standard client. Create a zk node and create a watch for its children. Producer can create nodes as children of this special node and connected clients (consumers) can receive them in an ordered manner. This setup will not affect the overall performance of the cluster, because the 'message bus' only uses observers to achieve its goal. 2) Write your custom Observer node by implementing ZK protocol that can receive and parse INFORM messages. These messages are consumed by Observers too. Obviously this solution could be enormously cumbersome, although Observers are the least complicated nodes of a ZK ensemble, but I'm not aware of any existing 'plugin' solution which would achieve similar forwarder mechanism. Regards, Andor On Wed, Jan 17, 2018 at 2:57 PM, Martin Gainty wrote: > > > ________________________________ > From: Bruno Freudensprung > Sent: Tuesday, January 16, 2018 7:54 AM > To: user@zookeeper.apache.org > Subject: Observers: a link to message bus? > > Hi, > > I have a question regarding this part of the Observers Guide ( > https://zookeeper.apache.org/doc/r3.4.11/zookeeperObservers.html): > ZooKeeper Observers zookeeperObservers.html> > zookeeper.apache.org > Setting up a ZooKeeper ensemble that uses Observers is very simple, and > requires just two changes to your config files. Firstly, in the config file > of every node ... > > > > "As a link to a message bus : [...] Observers would give a natural > integration point for this work: a plug-in mechanism could be used to > attach the stream of proposals an Observer sees to a publish-subscribe > system" > > I've not been able to find such a plug-in mechanism in the source code of > ZooKeeper. For the moment the only integration point I can think of is the > commitRequest method of the org.apache.zookeeper.server.quorum.ObserverZooKeeperServer > class. Am I looking in the right direction? > > To be honest: this is for the moment only pure curiosity since Apache > Kafka seems to be a more "natural" and out-of-the-box option for an ordered > message bus. However since I might probably need ZooKeeper for its > distributed lock and leader election features, I was wondering if I could > use it as an ordered message bus as well. > MG>if ActiveMQ does indeed support ordered messaging for multiple active > consumers thru Message Groups as stated: > MG>Message Groups which is the way to load balance multiple active > consumers listening to the same queue while preserving message order. > MG>i have asked why does Zookeeper not support MessageOrderingTechnology > such as ActiveMQ as a plugin? > MG>have you discovered "plugin integration" within the Zookeeper > Architecture? > > Best regards, > > Bruno. > MG>Thanks Bruno > --f4f5e808e28c9e11d1056382f2d3--