zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Shraer <shra...@gmail.com>
Subject Re: AW: Configuration management for zoo.cfg
Date Mon, 16 Dec 2019 00:23:19 GMT
Yes, that sounds like a good change to me.


Alex

On Sun, Dec 15, 2019 at 4:15 PM Aristedes Maniatis <ari@ish.com.au> wrote:

> How about a new property:
>
> dynamicConfigHistory=3
>
> which would preserve 3 historic config files. Or
>
> dynamicConfigHistory=0
>
> which would keep none.
>
>
> Does that sound like a reasonable approach? A default value of 0 would
> be what most people expect, although it is a change for people already
> wanting a folder full of files.
>
> I agree that dynamicConfigFile should point to the actual dynamic file,
> not sometimes to the real file and sometimes to the prefix of the real
> file. If there is any history worth keeping, then rolling them over log
> style (either with timestamps or config id) is a much more understood
> behaviour.
>
>
>
> Ari
>
>
>
> On 16/12/19 9:53am, Alexander Shraer wrote:
> > I wasn't sure whether extracting such information from the log is simple,
> > and since reconfigurations may impact the cluster in significant ways (or
> > in the extreme bring it down completely)
> > an easily accessible record seemed good to have, at least for debugging.
> I
> > agree that this can be made configurable, and would also not mind very
> much
> > not having a history at all, if others don't find it very useful.
> > However this is a breaking change so probably requires more people to
> chime
> > in.
> >
> >> In case of some network issue, where a node repeatedly flaps, why would
> > you want to fill the directory with possibly thousands of files?
> >
> > Automating reconfigations was not part of the release, only the basic
> > mechanism was provided and not for example the policy of when you'd want
> to
> > reconfigure and what changes to do.
> > But I agree that an automatic system like that should take care of this
> > situation.
> >
> >
> > Alex
> >
> >
> > On Sun, Dec 15, 2019 at 2:26 PM Aristedes Maniatis <ari@ish.com.au>
> wrote:
> >
> >> Can you explain a bit more about the use-case for when you'd want to
> >> keep the history of the dynamic file. Surely the log file will contain
> >> information about peers joining and leaving the cluster and is easier to
> >> parse if you care about tracking that sort of thing.
> >>
> >> In case of some network issue, where a node repeatedly flaps, why would
> >> you want to fill the directory with possibly thousands of files?
> >>
> >>
> >> Ari
> >>
> >>
> >> On 15/12/19 2:35pm, Alexander Shraer wrote:
> >>> Hi Ari,
> >>>
> >>> Yes, you're totally right about the design goals.
> >>>
> >>> A mode where historic files aren't preserved could be useful. This
> >>> could perhaps be added to the static config file as a parameter.
> >>>
> >>> Alternatively / in addition, maybe we could slightly change the way
> >> history
> >>> is staved. I don't really like the fact that we're actually using
> >>> the file name to determine the version of the config (rather than
> >>> information inside the file), this is used internally in ZK to decide
> >> which
> >>> config to use (the one with higher number wins).
> >>> This method could fix this issue as well:
> >>> - File name always stays the same, addressing your problem, and we
> don't
> >>> need to edit the static config file every time.
> >>> - Dynamic config file contains the config version as a key.
> >>> - Before overwriting the dynamic config file, we store a file with the
> >>> previous config, including the version in the file name.
> >>>
> >>> This would change the current behavior a bit, hopefully no one is
> relying
> >>> on the file name to contain the version.
> >>>
> >>> This should not be difficult to implement, would you like to open a
> Jira
> >>> and take a stab at implementing it ? I can review it.
> >>>
> >>> Something to notice about the "version" of the config - currently when
> >> the
> >>> config is stored in memory, it appears as a key in the configuration.
> >> When
> >>> stored in the temporary config file (pre-commit), it appears as an
> >> explicit
> >>> key, but when committed it does not appear inside the dynamic file -
> only
> >>> in the file name.This is controlled by the last argument of
> >>>    QuorumPeerConfig.writeDynamicConfig.
> >>>
> >>> See also QuorumPeerConfig.java parse() parseProperties() etc and
> >>> QuorumPeer.java setQuorumVerifier().
> >>>
> >>> Thanks,
> >>> Alex
> >>>
> >>> On Sat, Dec 14, 2019 at 6:32 PM Aristedes Maniatis <ari@ish.com.au>
> >> wrote:
> >>>> Will anything bad happen if I make the config file read-only for
> >>>> zookeeper? I assume the design goals here were:
> >>>>
> >>>> * atomic rewrites of the dynamic config, preserving historic files
> >>>>
> >>>> * ability for zookeeper to know which was the most recent config file
> on
> >>>> restart
> >>>>
> >>>>
> >>>> Those goals are a bit unnecessary for me. I don't really care about
> >>>> historic configuration, so just writing to a temp file and moving over
> >>>> the existing one would work great. Alternatively tracking the current
> >>>> file in memory without rewriting the zoo.cfg would also be great,
> since
> >>>> I don't care about the effort on startup to rediscover peers.
> >>>>
> >>>> Is there a way to get Zookeeper to play better with not rewriting its
> >>>> own config file for my use case?
> >>>>
> >>>>
> >>>> Ari
> >>>>
> >>>>
> >>>> On 12/12/19 5:53am, Alexander Shraer wrote:
> >>>>> It will change, the number represents the version of the
> configuration,
> >>>> and
> >>>>> will be updated if you issue a reconfiguration command. Its basically
> >> the
> >>>>> zxid of the command.
> >>>>>
> >>>>>
> >>>>> Alex
> >>>>>
> >>>>> On Tue, Dec 10, 2019 at 11:25 PM Aristedes Maniatis <ari@ish.com.au>
> >>>> wrote:
> >>>>>> On 11/12/19 6:21pm, Arne.Bachmann@dlr.de wrote:
> >>>>>>> Hey Ari,
> >>>>>>>
> >>>>>>> I directly used the filename   zoo.cfg.dynamic.100000000
   and
> never
> >>>>>> had a
> >>>>>>> problem.
> >>>>>>> Arne
> >>>>>> Hmmm... that's an oddly obvious answer. I just assumed the 100000000
> >>>>>> would change randomly. What's even the point of it?
> >>>>>>
> >>>>>>
> >>>>>> Ari
> >>>>>>
> >>>>>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message