zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enrico Olivelli <eolive...@gmail.com>
Subject Re: How to deliberately cause a split brain?
Date Fri, 19 Jun 2020 12:11:20 GMT
Il giorno ven 19 giu 2020 alle ore 13:40 Tim Ward <
tim.ward@featurespace.co.uk> ha scritto:

> From: Michael Han <hanm@apache.org>
> > There are different cases for split brain and how to test the monitor
> code depends on what signals
> > you are using - but most usually, a split brain case can be created by
> artificially split two quorum
> > out of a single quorum through manual configuration change (e.g. a 7
> quorum servers can be split
> > into a 4 quorum with a 3 quorum - just exclude the servers from each
> config).
>
> Yes, I am coming to the conclusion that one could in effect
>
> (a) build two separate Zookeeper ensembles
> (b) point parts of one's application at one ensemble and the rest at the
> other
>
> and expect the application to misbehave somewhat horribly.
>

I don't know your application, but the application will behave weirdly from
the beginning.
If you use "locks", than they won't work, if you run leader election...you
will have two leaders from the beginning

Usually applications have some other out-of-bound (in respect to ZK)
coordination mechanisms that prevents misbehaviour
like using Compare-And-Set operations and versioning on Database records
for critical state
so hopefully you will detect some problem


>
> And I've come up with some stuff I can monitor to detect all that in
> various ways.
>

Awesome, do you mean a "generic" tool ?


Enrico


>
> *BUT* this detection process would reply on the person who had carefully
> and cunningly maliciously mis-configured Zookeeper at the same time
> *correctly* configuring the monitoring system. How likely is that, I
> wonder? So I'm discussing with my customer how much use this feature is
> really likely to be in the real world ...
>
> Thanks all for the various comments.
>
> Tim Ward
> This message, and any files/attachments transmitted together with it, is
> intended for the use only of the person (or persons) to whom it is
> addressed. It may contain information which is confidential and/or
> protected by legal privilege. Accordingly, any dissemination, distribution,
> copying or use of this message, or any part of it or anything sent together
> with it, other than by intended recipients, may constitute a breach of
> civil or criminal law and is hereby prohibited. Unless otherwise stated,
> any views expressed in this message are those of the person sending it and
> not the sender's employer. No responsibility, legal or otherwise, of
> whatever nature, is accepted as to the accuracy of the contents of this
> message or for the completeness of the message as received. Anyone who is
> not the intended recipient of this message is advised to make no use of it
> and is requested to contact Featurespace Limited as soon as possible. Any
> recipient of this message who has knowledge or suspects that it may have
> been the subject of unauthorised interception or alteration is also
> requested to contact Featurespace Limited.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message