From user-return-14555-archive-asf-public=cust-asf.ponee.io@storm.apache.org Fri Aug 23 08:57:39 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 64A21180637 for ; Fri, 23 Aug 2019 10:57:39 +0200 (CEST) Received: (qmail 21044 invoked by uid 500); 23 Aug 2019 08:57:38 -0000 Mailing-List: contact user-help@storm.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@storm.apache.org Delivered-To: mailing list user@storm.apache.org Received: (qmail 21033 invoked by uid 99); 23 Aug 2019 08:57:38 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Aug 2019 08:57:38 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id A93EB1A40B2 for ; Fri, 23 Aug 2019 08:57:37 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.8 X-Spam-Level: * X-Spam-Status: No, score=1.8 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id HvMzw2Q3jD16 for ; Fri, 23 Aug 2019 08:57:35 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::242; helo=mail-oi1-x242.google.com; envelope-from=igor.fraa@gmail.com; receiver= Received: from mail-oi1-x242.google.com (mail-oi1-x242.google.com [IPv6:2607:f8b0:4864:20::242]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id D6F9E7DD38 for ; Fri, 23 Aug 2019 08:57:34 +0000 (UTC) Received: by mail-oi1-x242.google.com with SMTP id y8so6468840oih.10 for ; Fri, 23 Aug 2019 01:57:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=J5Dcq5dLUgF6yQhdLgUzxmTehxHC0ScQTUedSHSVCxI=; b=TqygmpHeFQ6sdeA3wezIT4pJwlPqhJe/hXAEOv+VqMuFl3PBfY6a2bVtlfVPEYt/Bh sC1HaT/C9UOKC9kFYSIi8Mc5fGebgOOZCu+EBYct4tJthF0p9kwEyTERuLT1S501vzlw Jn+6xsjpe9LAFNtvyhrS14Qwim1CaEjWfpfA5mdkKg+qHiLhSvQiUPJMy0EteCiHt3Q+ gy+7KPJtOEkMEIkzmKzpGvsJpRGmd/OMMt9qSY43yPvmG8ub44t9vLcuYB+SqS0D23t+ rvFCr70CwaBFTKxkIa7bLMIe0KZyoGTht0fPgsXpVIdKml5gj4wKzhJBRIcEINNFJDKg kTFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=J5Dcq5dLUgF6yQhdLgUzxmTehxHC0ScQTUedSHSVCxI=; b=X6O8Uq2FKOQ+ZMxvJuORv1LNAAf1q2ABnp6sorkkFKvYRRaxWUvQll8fwl2WN/Pw3k t+qfDErYieC6e7ufND9i1FPWBu4ehod8vN5nEcRxhHNFR7Qpu1Xg9MLw1OHc2JaFEWuw r+1IZZssm++ixcz/5RqytOsVmdR5lq5liK9C7c0jmR593GFno+GpHXUJaQ+FrEDGLDE5 oJ/jwNdXNylZGZ4fUWcncJl/iLV9bWuHbLtFOl+1Yoq5TNevqQ9/Xkb/zEwqESCh9ieR l3IqLsOwumx/tq9xKcQ9yV7iO9CA8nfH49FchFkTv3o3QBJRnrDtM98dL071hn7qTBsx eD2Q== X-Gm-Message-State: APjAAAWrDf8TpzY6TxCyqfmuXSkAFAoM7HOSMmsfMeimp6+zixmBe7I7 jQ1MyhtyaoCxkQ4UTbsLDh4sWGMH1nCIjTdqjMR7PBhP X-Google-Smtp-Source: APXvYqx7+9Xuprtjij3rIbb1oSBAosOBPODJ9cAGoFxwUHMB4/DG4IJW4T1oT2z5JjhBIYAKGeBA7ro7bxYm8QxDvCc= X-Received: by 2002:aca:5082:: with SMTP id e124mr2325879oib.52.1566550653186; Fri, 23 Aug 2019 01:57:33 -0700 (PDT) MIME-Version: 1.0 References: <6E52DC64-7BB9-4BF9-9998-643C5C5C7EBD@gmail.com> In-Reply-To: <6E52DC64-7BB9-4BF9-9998-643C5C5C7EBD@gmail.com> From: "Igor A." Date: Fri, 23 Aug 2019 11:57:20 +0300 Message-ID: Subject: Re: pacemaker question To: user@storm.apache.org Content-Type: multipart/alternative; boundary="00000000000034b5f80590c4ffbc" --00000000000034b5f80590c4ffbc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Ethan, >Pacemaker is an additional daemon that takes some load from Zookeeper so it will not be overloaded, which means Zookeeper is still required. could you please confirm that Zookeeper is required in any Storm configuration? On Thu, Aug 22, 2019, 19:06 Ethan Li wrote: > Hi Ignor, > > If you are running 2.0.0 or higher version of topology on 2.x Storm > cluster, thanks to the changes of > https://issues.apache.org/jira/browse/STORM-2693, > https://github.com/apache/storm/pull/2433, Pacemaker is no longer needed. > It won=E2=80=99t do anything. > > > If you are running older version of topology on 2.x Storm cluster, > Pacemaker is an additional daemon that takes some load from Zookeeper so > it will not be overloaded, which means Zookeeper is still required. > https://github.com/apache/storm/blob/master/docs/Pacemaker.md. So you > will see supervisors connecting to zookeeper. > > storm.zookeeper.port is used to connect to zookeeper so you want to set i= t > as your zookeeper port, default to 2181. > https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apac= he/storm/Config.java#L1067 > > > Thanks > Ethan > > > On Aug 22, 2019, at 10:24 AM, Igor A. wrote: > > Hello all, > > I'm trying to run Storm version 2.0.0 with Pacemaker. > I've set up everything like it is described in documentation (there is no= t > so much parameters to miss something). Pacemaker is listening on default > :6699. > The issue is despite the fact storm.cluster.state.store is set to > "[...].PaceMakerStateStorageFactory", Storm Supervisor tries to connect t= o > Zookeeper on :2181. > > I can see in logs that Storm launched Zookeeper client and it runs in 3.4 > compatibility mode. > I tried to set storm.zookeeper.port to 6699, but this caused Pacemaker to > die with an NPE. Note: after reading sources i've found that > PacemakerClient doesn't use zookeeper.port > Does anyone know how to make Storm communicate with Pacemaker? May be > somebody have a working storm.yaml to enable Pacemake Client? > > > > > --00000000000034b5f80590c4ffbc Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Ethan,=C2=A0

>Pacemaker is a= n additional daemon that takes some load from Zookeeper so it will not be o= verloaded, which means Zookeeper is still required.=C2=A0

could you please confirm that Zookee= per is required in any Storm configuration?

On Thu, Aug 22, 2019= , 19:06 Ethan Li <ethanopen= source@gmail.com> wrote:
Hi Ignor,
If you are running 2.0.0 or higher version of topology on 2= .x Storm cluster, thanks to the changes of =C2=A0htt= ps://issues.apache.org/jira/browse/STORM-2693,=C2=A0htt= ps://github.com/apache/storm/pull/2433, Pacemaker is no longer needed. = It won=E2=80=99t do anything.=C2=A0


If you are running older version of topology on 2.x Storm cluster, =C2=A0P= acemaker is an additional daemon that takes some load from Zookeeper so it = will not be overloaded, which means Zookeeper is still required. =C2=A0https://github.com/apache/storm/blob/master= /docs/Pacemaker.md. So you will see supervisors connecting to zookeeper= .=C2=A0

storm.zookeeper.port is used to conne= ct to zookeeper so you want to set it as your zookeeper port, default to 21= 81.=C2=A0https://github.com/apache/storm/blob/master/storm-client/src/jvm/or= g/apache/storm/Config.java#L1067


Thanks
Ethan

<= span>
On Aug 22, 2019, at 10:24 AM, Igor A. <igor.fraa@gmail.com>= ; wrote:

Hello all,

I'm trying to run Storm version 2.0.0 with Pacemaker.=
I've set up everything like it is described in docum= entation (there is not so much parameters to miss something). Pacemaker is = listening on default :6699.
The issue is despite the fact= storm.cluster.state.store is set to "[...].PaceMakerStateStorageFacto= ry", Storm Supervisor tries to connect to Zookeeper on :2181.

I can see in logs that Storm launched Zook= eeper client and it runs in 3.4 compatibility mode.
I tri= ed to set storm.zookeeper.port to 6699, but this caused Pacemaker to die wi= th an NPE. Note: after reading sources i've found that PacemakerClient = doesn't use zookeeper.port
Does anyone know how to make Storm c= ommunicate with Pacemaker? May be somebody have a working storm.yaml to ena= ble Pacemake Client?




--00000000000034b5f80590c4ffbc--