From user-return-12944-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Wed Sep 23 09:04:23 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mailroute1-lw-us.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 01C23180660 for ; Wed, 23 Sep 2020 11:04:23 +0200 (CEST) Received: from mail.apache.org (localhost [127.0.0.1]) by mailroute1-lw-us.apache.org (ASF Mail Server at mailroute1-lw-us.apache.org) with SMTP id D6692124931 for ; Wed, 23 Sep 2020 09:04:21 +0000 (UTC) Received: (qmail 24158 invoked by uid 500); 23 Sep 2020 09:04:21 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 24142 invoked by uid 99); 23 Sep 2020 09:04:20 -0000 Received: from spamproc1-he-de.apache.org (HELO spamproc1-he-de.apache.org) (116.203.196.100) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Sep 2020 09:04:20 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-de.apache.org (ASF Mail Server at spamproc1-he-de.apache.org) with ESMTP id 045641FF39B for ; Wed, 23 Sep 2020 09:04:20 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-de.apache.org X-Spam-Flag: NO X-Spam-Score: -0.002 X-Spam-Level: X-Spam-Status: No, score=-0.002 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamproc1-he-de.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([116.203.227.195]) by localhost (spamproc1-he-de.apache.org [116.203.196.100]) (amavisd-new, port 10024) with ESMTP id JEKqbgzF8ctJ for ; Wed, 23 Sep 2020 09:04:18 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.167.178; helo=mail-oi1-f178.google.com; envelope-from=eolivelli@gmail.com; receiver= Received: from mail-oi1-f178.google.com (mail-oi1-f178.google.com [209.85.167.178]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 3CF69BC24B for ; Wed, 23 Sep 2020 09:04:18 +0000 (UTC) Received: by mail-oi1-f178.google.com with SMTP id 185so24237695oie.11 for ; Wed, 23 Sep 2020 02:04:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=lYy69/oKUWXtBT6lQoLbv1ckJh01VHPe2ej0JJlgyoY=; b=cFs/YtCit6c1qIPPRWPqvQma43fBC/0K8/ef8o2j3SLAdaMhAa/D35r67EE3RBoVR1 QDYc+6en5JNl8DkXBLscxw/U51uMpWJFLc+f1Dc//olZqhaXC6bennKds1RFveXIop5t 9e8QLJQilAqBciOXEngCJRYMTJ8YVUNsk3/G1G2TKLjfI+vm4NJ3saLF6YRMQdVFiUCQ uRIYSkOBQQ1AsxHDjHaNRPEmkxd1C8OB4vk3hA0SpazYSBoYEvxZ8A/kIMHPZOrvoXmu bVt1Engqjv8IafpetkzssMVeTT1gwSwg8uKXm/Lz8PygCFfTOQKxFkaFPFJxPtJJ2bQ0 EZSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=lYy69/oKUWXtBT6lQoLbv1ckJh01VHPe2ej0JJlgyoY=; b=rqVKPveIPbOHMhrDoQiI7TYg13YOty+ybelxGHUclYFGXahTzbESfqfdE3ULoun/kc uTRR54obw3HD9Wb45WzAYq3t/w3gZVIbW3PhQ1w9UKovvEE1mEM7O1enpR06e8GE8SxD DjRHm66m4S8Wak/RvbEsDLk6NHU/B100h78/8KR+DBjrgFlYMQkHisNjF6hewmDQbCvx C4e2hZX/d4OtDRA6oUAC1ErUDUwg8BGWk4X6vi5w6zQdxgvXPjuThynLfZXNVhy6Ud9+ KCeI3XSYP5j0CdcGG4brNgEyD5DPMzT42cgw3gOAhj5FhmysPNS4gkyL5dm7cOMoR2Z/ Xdog== X-Gm-Message-State: AOAM530Oit6mp/CjBH3LbXJT1Q4syxXeydtaLA6k11gxv44W2Zlwk4pH 6FETLQ7hIENkIkCJNu5B2Cf4YFxFGaCyDTQ3WOgzmipYVzqYaQ== X-Google-Smtp-Source: ABdhPJw7fMWMbbX+WLUSNvkFnXoNa4O7b4/dfFRs6W13vRRRWck3Z4qkls1+ftDrtz39LIybkCX6d+DpAlJQWWql7ys= X-Received: by 2002:a05:6808:54:: with SMTP id v20mr4893524oic.50.1600851851789; Wed, 23 Sep 2020 02:04:11 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Enrico Olivelli Date: Wed, 23 Sep 2020 11:04:00 +0200 Message-ID: Subject: Re: ZooKeeper Cluster Health Checking To: UserZooKeeper Content-Type: multipart/alternative; boundary="000000000000f6ca5b05aff75de1" --000000000000f6ca5b05aff75de1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Adrien Il giorno mer 23 set 2020 alle ore 10:59 adrien ruffie < adriennolarsen@hotmail.fr> ha scritto: > Hello all, > > I have a problem in production ... > > We have the following zoo configuration file: > > tickTime=3D4000 > dataDir=3D/var/lib/zookeeper > > dataLogDir=3D/var/log/zookeeper > > initLimit=3D30 > syncLimit=3D15 > > autopurge.snapRetainCount=3D3 > autopurge.purgeInterval=3D48 > > clientPort=3D2181 > maxClientCnxns=3D60 > > server.1=3DZOO1:2888:3888 > server.2=3DZOO2:2888:3888 > server.3=3DZOO3:2888:3888 > server.4=3DZOO4:2888:3888 > server.5=3DZOO5:2888:3888 > > We are in zookeeper-3.4.10, but we recently saw, that log and snapshot > aren't purge ... > do you know this issue, is a bug, or bad configuration ? > Do you see errors in logs ? Are you using standard Apache distributions? Enrico > > Thank you very much and best regards > > Adrien Ruffi=C3=A9 > ________________________________ > De : adrien ruffie > Envoy=C3=A9 : mercredi 18 juillet 2018 09:01 > =C3=80 : user@zookeeper.apache.org > Objet : RE: ZooKeeper Cluster Health Checking > > Ok thank Harish, > > I keep the idea ! > > > Best regards, > > > Adrien > > ________________________________ > De : harish lohar > Envoy=C3=A9 : mardi 17 juillet 2018 23:13:28 > =C3=80 : user@zookeeper.apache.org > Objet : Re: ZooKeeper Cluster Health Checking > > We did it via java monitoring app , using zookeeper java api which sends = 4 > lw commands to zookeeper and returns the output. > > > Thanks > Harish > > On Tue, Jul 17, 2018 at 2:00 AM adrien ruffie > wrote: > > > Hi Harish, > > > > > > thank you very much for this advise and explanation ! > > > > Do you think with just a simple script shell for checking all this > metrics > > is enough ? Or would better to do it in a Java with a simple monitoring > > application? > > > > > > Thank again, > > > > > > Best regards, > > > > > > Adrien > > > > ________________________________ > > De : harish lohar > > Envoy=C3=A9 : mardi 17 juillet 2018 04:13:51 > > =C3=80 : user@zookeeper.apache.org > > Objet : Re: ZooKeeper Cluster Health Checking > > > > Hi Adrian, > > Below zookeeper commands are generally used to get health of zookeeper > > cluster > > stat > > > > Lists brief details for the server and connected clients. > > > > usage echo stat | nc server port > > > > This gives whether cluster is up /down. If down this will give that > > > > Zookeeper instance is currently not serving any request - which means > > either the leader election is failing or <=3D 50% of zookeeper node in > > cluster are down. > > > > > > mntr > > > > *New in 3.4.0:* Outputs a list of variables that could be used for > > monitoring the health of the cluster. > > > > $ echo mntr | nc localhost 2185 > > > > zk_version 3.4.0 > > zk_avg_latency 0 > > zk_max_latency 0 > > zk_min_latency 0 > > zk_packets_received 70 > > zk_packets_sent 69 > > zk_outstanding_requests 0 > > zk_server_state leader > > zk_znode_count 4 > > zk_watch_count 0 > > zk_ephemerals_count 0 > > zk_approximate_data_size 27 > > zk_followers 4 - only exposed by the Leader > > zk_synced_followers 4 - only exposed by the Leader > > zk_pending_syncs 0 - only exposed by the Leader > > zk_open_file_descriptor_count 23 - only available on Unix platforms > > zk_max_file_descriptor_count 1024 - only available on Unix platforms > > > > The output is compatible with java properties format and the content ma= y > > change over time (new keys added). Your scripts should expect changes. > > > > ATTENTION: Some of the keys are platform specific and some of the keys > are > > only exported by the Leader. > > > > The output contains multiple lines with the following format: > > > > > > On Mon, Jul 16, 2018 at 10:13 AM adrien ruffie < > adriennolarsen@hotmail.fr> > > wrote: > > > > > Hello all, > > > > > > > > > In my company we have a Zookeeper production cluster. > > > > > > > > > But we don't really know how can we check the health of our cluster..= . > > > > > > > > > Can we advise us about this topic ? > > > > > > > > > I know this topic may has been cropping up for a while, but I don't > > really > > > found any concrete solution. > > > > > > > > > Do you use a monitoring tools ? Which can launch alert ? > > > > > > What metrics/properties/any thing which can indicate that our cluster > > > isn't in good health. > > > > > > > > > Thank you very much and best regards > > > > > > > > > Adrien > > > > > > --000000000000f6ca5b05aff75de1--