zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Szalay-Bekő Máté <szalay.beko.m...@gmail.com>
Subject Re: How to deliberately cause a split brain?
Date Thu, 18 Jun 2020 08:54:29 GMT
Even with using different server configs, no vote from "unknown" server
should be accepted by the others. (at least if dynamic reconfig is
disabled)

However, I never really tested with intentionally bad configs. There might
exist some bugs we are unaware of and haven't fixed yet. There was an
important bugfix recently which hasn't been released yet:
https://github.com/apache/zookeeper/pull/1356 - it is always possible that
we still have holes somewhere. But still, I think any such scenario that
leads to a split-brain should be considered as a bug.

On Thu, Jun 18, 2020 at 10:43 AM Szalay-Bekő Máté <
szalay.beko.mate@gmail.com> wrote:

> > You can just use iptables and simulate that some network paths are not
> working properly
> I would assume ZooKeeper would handle this, and if no partition has
> quorum, then no writes will be enabled / no leader will be present.
> (readonly mode can still work, if configured properly)
>
> Honestly, I can not really think of any way to get split-brain. If someone
> can, then let us know so that we can fix it ;)
>
> Cheers,
> Mate
>
> On Thu, Jun 18, 2020 at 10:34 AM Enrico Olivelli <eolivelli@gmail.com>
> wrote:
>
>> Tim
>> You can just use iptables and simulate that some network paths are not
>> working properly
>>
>> Enrico
>>
>> Il Gio 18 Giu 2020, 10:26 Tim Ward <tim.ward@featurespace.co.uk> ha
>> scritto:
>>
>> > I have been tasked with writing some monitoring code to detect a
>> Zookeeper
>> > split brain using the monitoring system.
>> >
>> >
>> >
>> > All well and good, I think I can see how to do that, but what about
>> > testing? How can I deliberately provoke a Zookeeper ensemble into going
>> > into a split brain state so that I can test the detection code?
>> >
>> >
>> >
>> > Scenarios might be along the lines of: the ops people want to increase
>> an
>> > ensemble size, but get something wrong in the (manual) writing of the
>> > configuration files or in the (manual) restarting of instances. What
>> > "something" is likely to work here?
>> >
>> >
>> >
>> >
>> >
>> > *Tim Ward*
>> >
>> >
>> >
>> >
>> >
>> > Principal engineer
>> >
>> >
>> >
>> >
>> >
>> > —
>> >
>> >
>> >
>> >
>> >
>> > *t: *+44 (0) 1223 345 940
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > *Broers Building, Hauser Forum*
>> >
>> > *21 JJ Thomson Avenue, Cambridge*
>> >
>> > *CB3 0FA United Kingdom*
>> >
>> >
>> >
>> >
>> >
>> > featurespace.com <https://www.featurespace.com/>* | *Twitter
>> > <https://twitter.com/FeaturespaceLtd>* | *LinkedIn
>> > <https://www.linkedin.com/company/featurespace>
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > [image: A screenshot of a cell phone Description automatically
>> generated]
>> >
>> >
>> >
>> >
>> > This message, and any files/attachments transmitted together with it, is
>> > intended for the use only of the person (or persons) to whom it is
>> > addressed. It may contain information which is confidential and/or
>> > protected by legal privilege. Accordingly, any dissemination,
>> distribution,
>> > copying or use of this message, or any part of it or anything sent
>> together
>> > with it, other than by intended recipients, may constitute a breach of
>> > civil or criminal law and is hereby prohibited. Unless otherwise stated,
>> > any views expressed in this message are those of the person sending it
>> and
>> > not the sender's employer. No responsibility, legal or otherwise, of
>> > whatever nature, is accepted as to the accuracy of the contents of this
>> > message or for the completeness of the message as received. Anyone who
>> is
>> > not the intended recipient of this message is advised to make no use of
>> it
>> > and is requested to contact Featurespace Limited as soon as possible.
>> Any
>> > recipient of this message who has knowledge or suspects that it may have
>> > been the subject of unauthorised interception or alteration is also
>> > requested to contact Featurespace Limited.
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message