zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Shraer <shra...@gmail.com>
Subject Re: AW: Configuration management for zoo.cfg
Date Sun, 15 Dec 2019 22:59:32 GMT
Another potential advantage is that if something bad happened, you would
have the latest working config readily available.

On Sun, Dec 15, 2019 at 2:53 PM Alexander Shraer <shralex@gmail.com> wrote:

> I wasn't sure whether extracting such information from the log is simple,
> and since reconfigurations may impact the cluster in significant ways (or
> in the extreme bring it down completely)
> an easily accessible record seemed good to have, at least for debugging. I
> agree that this can be made configurable, and would also not mind very much
> not having a history at all, if others don't find it very useful.
> However this is a breaking change so probably requires more people to
> chime in.
>
> > In case of some network issue, where a node repeatedly flaps, why would
> you want to fill the directory with possibly thousands of files?
>
> Automating reconfigations was not part of the release, only the basic
> mechanism was provided and not for example the policy of when you'd want to
> reconfigure and what changes to do.
> But I agree that an automatic system like that should take care of this
> situation.
>
>
> Alex
>
>
> On Sun, Dec 15, 2019 at 2:26 PM Aristedes Maniatis <ari@ish.com.au> wrote:
>
>> Can you explain a bit more about the use-case for when you'd want to
>> keep the history of the dynamic file. Surely the log file will contain
>> information about peers joining and leaving the cluster and is easier to
>> parse if you care about tracking that sort of thing.
>>
>> In case of some network issue, where a node repeatedly flaps, why would
>> you want to fill the directory with possibly thousands of files?
>>
>>
>> Ari
>>
>>
>> On 15/12/19 2:35pm, Alexander Shraer wrote:
>> > Hi Ari,
>> >
>> > Yes, you're totally right about the design goals.
>> >
>> > A mode where historic files aren't preserved could be useful. This
>> > could perhaps be added to the static config file as a parameter.
>> >
>> > Alternatively / in addition, maybe we could slightly change the way
>> history
>> > is staved. I don't really like the fact that we're actually using
>> > the file name to determine the version of the config (rather than
>> > information inside the file), this is used internally in ZK to decide
>> which
>> > config to use (the one with higher number wins).
>> > This method could fix this issue as well:
>> > - File name always stays the same, addressing your problem, and we don't
>> > need to edit the static config file every time.
>> > - Dynamic config file contains the config version as a key.
>> > - Before overwriting the dynamic config file, we store a file with the
>> > previous config, including the version in the file name.
>> >
>> > This would change the current behavior a bit, hopefully no one is
>> relying
>> > on the file name to contain the version.
>> >
>> > This should not be difficult to implement, would you like to open a Jira
>> > and take a stab at implementing it ? I can review it.
>> >
>> > Something to notice about the "version" of the config - currently when
>> the
>> > config is stored in memory, it appears as a key in the configuration.
>> When
>> > stored in the temporary config file (pre-commit), it appears as an
>> explicit
>> > key, but when committed it does not appear inside the dynamic file -
>> only
>> > in the file name.This is controlled by the last argument of
>> >   QuorumPeerConfig.writeDynamicConfig.
>> >
>> > See also QuorumPeerConfig.java parse() parseProperties() etc and
>> > QuorumPeer.java setQuorumVerifier().
>> >
>> > Thanks,
>> > Alex
>> >
>> > On Sat, Dec 14, 2019 at 6:32 PM Aristedes Maniatis <ari@ish.com.au>
>> wrote:
>> >
>> >> Will anything bad happen if I make the config file read-only for
>> >> zookeeper? I assume the design goals here were:
>> >>
>> >> * atomic rewrites of the dynamic config, preserving historic files
>> >>
>> >> * ability for zookeeper to know which was the most recent config file
>> on
>> >> restart
>> >>
>> >>
>> >> Those goals are a bit unnecessary for me. I don't really care about
>> >> historic configuration, so just writing to a temp file and moving over
>> >> the existing one would work great. Alternatively tracking the current
>> >> file in memory without rewriting the zoo.cfg would also be great, since
>> >> I don't care about the effort on startup to rediscover peers.
>> >>
>> >> Is there a way to get Zookeeper to play better with not rewriting its
>> >> own config file for my use case?
>> >>
>> >>
>> >> Ari
>> >>
>> >>
>> >> On 12/12/19 5:53am, Alexander Shraer wrote:
>> >>> It will change, the number represents the version of the
>> configuration,
>> >> and
>> >>> will be updated if you issue a reconfiguration command. Its basically
>> the
>> >>> zxid of the command.
>> >>>
>> >>>
>> >>> Alex
>> >>>
>> >>> On Tue, Dec 10, 2019 at 11:25 PM Aristedes Maniatis <ari@ish.com.au>
>> >> wrote:
>> >>>> On 11/12/19 6:21pm, Arne.Bachmann@dlr.de wrote:
>> >>>>> Hey Ari,
>> >>>>>
>> >>>>> I directly used the filename   zoo.cfg.dynamic.100000000   
and
>> never
>> >>>> had a
>> >>>>> problem.
>> >>>>> Arne
>> >>>> Hmmm... that's an oddly obvious answer. I just assumed the 100000000
>> >>>> would change randomly. What's even the point of it?
>> >>>>
>> >>>>
>> >>>> Ari
>> >>>>
>> >>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message