From user-return-12442-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Sun Dec 15 03:36:00 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 2A8AD180629 for ; Sun, 15 Dec 2019 04:36:00 +0100 (CET) Received: (qmail 79560 invoked by uid 500); 15 Dec 2019 03:35:58 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 79548 invoked by uid 99); 15 Dec 2019 03:35:57 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 15 Dec 2019 03:35:57 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 52DBAC017A for ; Sun, 15 Dec 2019 03:35:57 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.001 X-Spam-Level: X-Spam-Status: No, score=-0.001 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id LoduhBm7i2jC for ; Sun, 15 Dec 2019 03:35:54 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.208.182; helo=mail-lj1-f182.google.com; envelope-from=shralex@gmail.com; receiver= Received: from mail-lj1-f182.google.com (mail-lj1-f182.google.com [209.85.208.182]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 47AD5BC509 for ; Sun, 15 Dec 2019 03:35:54 +0000 (UTC) Received: by mail-lj1-f182.google.com with SMTP id e10so3012699ljj.6 for ; Sat, 14 Dec 2019 19:35:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=hk4ClM9PUkOHzj8fBib/7Cqu5/3ws5NpU6TF/R+hpTk=; b=A71iyLWPoG2+7Cnm6Ch5L5YG5KtofDGGbvVi1KpDgrdR+BOd7UZyOAJOaNj0z3BPPa oNHQgxc3rgOC02aBUpjORkjD1x76HQdjVM7nQ9fmAhExYs0OQrIirnxPVx3nn1/2U61i Q3ycDXauEf5YvXU+nk+FseekWYEFqrSbxmtKPN+/ZXt1poX34X2wLQlTnlKjt/ztWfxt 1dnVNgg3wxpOFiMfbebbaTZvS69KZwFXvNG49A4H/EsIGI2fiWgQ5ejnx6Rp4ab7qyto UMIHXFZSEuL2eEIvVWeirCSqxQkKPww50iK1ugSp8N9Akvgq0dVxJCuNp9/Axdym+f3C NotA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=hk4ClM9PUkOHzj8fBib/7Cqu5/3ws5NpU6TF/R+hpTk=; b=CjFulpyigpS8yRNXjk6JTf22LE6jA1xude5HMzLenSanzP98Qga9lqAJG5l/2mPJma SLw5G+LWizrq0P1mz9E7yvKaQEU2K1HVAoMlbAodxLoa3oHYQJzl3L/iwVFoeTh6uwAm vDRiPiqzOS1nrFK57pNsmnFJytg3uo8IfMFl+Cquvlvi9fuTqWfNOMMuOkgxC3aPZSmf 8uaBiLZ14RcvppWPrtIYpxNNCPQZUnMO+ZkABtbGwJe82xfUeLH2+X9XVC8XcBcFPwru zB76Mkg3aPdZjRbA2t4rXEt2yalpOd30BUze5KrjoMYqGN7ueRB6odw56ZmXVSucYjGT rGFg== X-Gm-Message-State: APjAAAWqunGWiEw0S3HtJEK6kJEbxdxSp5KhoVIKpw4HUzT50b6v89fO JcPeFD2JAEyPKSTwH2o4HBe/Rsb65M/PsFOhVEHqyyk4 X-Google-Smtp-Source: APXvYqxGu4mf2FxXjw9SMvXVSPmxy/GWAg6CPI7L7oXnJT449sr87MhSLpjdf91hgQO5YroaBkeVnsiDy2F5TRw24TQ= X-Received: by 2002:a2e:9842:: with SMTP id e2mr15510161ljj.101.1576380952511; Sat, 14 Dec 2019 19:35:52 -0800 (PST) MIME-Version: 1.0 References: <7a4d6b9e-27c1-6e57-462e-286d04984766@ish.com.au> <048f3e17be5145e39e09ca421905c307@dlr.de> <7996f33a-250b-d679-8bc9-9091450b4e6e@ish.com.au> In-Reply-To: <7996f33a-250b-d679-8bc9-9091450b4e6e@ish.com.au> From: Alexander Shraer Date: Sat, 14 Dec 2019 19:35:41 -0800 Message-ID: Subject: Re: AW: Configuration management for zoo.cfg To: user@zookeeper.apache.org Content-Type: multipart/alternative; boundary="000000000000b48ab40599b5ca97" --000000000000b48ab40599b5ca97 Content-Type: text/plain; charset="UTF-8" Hi Ari, Yes, you're totally right about the design goals. A mode where historic files aren't preserved could be useful. This could perhaps be added to the static config file as a parameter. Alternatively / in addition, maybe we could slightly change the way history is staved. I don't really like the fact that we're actually using the file name to determine the version of the config (rather than information inside the file), this is used internally in ZK to decide which config to use (the one with higher number wins). This method could fix this issue as well: - File name always stays the same, addressing your problem, and we don't need to edit the static config file every time. - Dynamic config file contains the config version as a key. - Before overwriting the dynamic config file, we store a file with the previous config, including the version in the file name. This would change the current behavior a bit, hopefully no one is relying on the file name to contain the version. This should not be difficult to implement, would you like to open a Jira and take a stab at implementing it ? I can review it. Something to notice about the "version" of the config - currently when the config is stored in memory, it appears as a key in the configuration. When stored in the temporary config file (pre-commit), it appears as an explicit key, but when committed it does not appear inside the dynamic file - only in the file name.This is controlled by the last argument of QuorumPeerConfig.writeDynamicConfig. See also QuorumPeerConfig.java parse() parseProperties() etc and QuorumPeer.java setQuorumVerifier(). Thanks, Alex On Sat, Dec 14, 2019 at 6:32 PM Aristedes Maniatis wrote: > Will anything bad happen if I make the config file read-only for > zookeeper? I assume the design goals here were: > > * atomic rewrites of the dynamic config, preserving historic files > > * ability for zookeeper to know which was the most recent config file on > restart > > > Those goals are a bit unnecessary for me. I don't really care about > historic configuration, so just writing to a temp file and moving over > the existing one would work great. Alternatively tracking the current > file in memory without rewriting the zoo.cfg would also be great, since > I don't care about the effort on startup to rediscover peers. > > Is there a way to get Zookeeper to play better with not rewriting its > own config file for my use case? > > > Ari > > > On 12/12/19 5:53am, Alexander Shraer wrote: > > It will change, the number represents the version of the configuration, > and > > will be updated if you issue a reconfiguration command. Its basically the > > zxid of the command. > > > > > > Alex > > > > On Tue, Dec 10, 2019 at 11:25 PM Aristedes Maniatis > wrote: > > > >> On 11/12/19 6:21pm, Arne.Bachmann@dlr.de wrote: > >>> Hey Ari, > >>> > >>> I directly used the filename zoo.cfg.dynamic.100000000 and never > >> had a > >>> problem. > >>> Arne > >> > >> Hmmm... that's an oddly obvious answer. I just assumed the 100000000 > >> would change randomly. What's even the point of it? > >> > >> > >> Ari > >> > >> > --000000000000b48ab40599b5ca97--