Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <CABDss3j-WscFpWUSdR5pJnfd8_H72pU0COq0ytCFyPpHvU-N0Q@mail.gmail.com>
References: <CAHKVN8gHrFcRZDSp+ZbQ_GJbY1AZ4ajRAUasvpGjjWzd5YhhKQ@mail.gmail.com>
 <DEC4CB83-C883-4B18-BAB2-375522B0A8EB@apache.org> <CABDss3hf0mv6vo9tv2Oc=uNgZrs-UWLPJnH99HD=s70hYT+6ew@mail.gmail.com>
 <CAGcMBHh9cdj-2uOrCkUYSsGDRddk7gigz_Xu7YTbwSSxj6FJDA@mail.gmail.com>
 <563bee65-0268-4d50-bb51-b22ba42838f0@apache.org> <CAGcMBHj1mcc6HGVjDw5-SKsR-EM_kFVHRrkRg25iKfcwdE5oHw@mail.gmail.com>
 <003f9622-c5dd-46fd-8f87-a61c631a8375@apache.org> <CAHKVN8gPf43e61k7w5tqboDPdUPsFsQJUDJejm6Wa1tWSxvCAQ@mail.gmail.com>
 <CAGcMBHg0uJDw-LmG0ZkVf_dVD3N23VOHGrZRAJ5ELjxs07ocRw@mail.gmail.com> <CABDss3j-WscFpWUSdR5pJnfd8_H72pU0COq0ytCFyPpHvU-N0Q@mail.gmail.com>
From: Sergey Chugunov <sergey.chugunov@gmail.com>
Date: Thu, 3 Aug 2017 16:46:25 +0300
Message-ID: <CAHKVN8i2ZNKHmnvfx7j3reZpZuZDtGeAhE32NwLMMF0YgGa9pQ@mail.gmail.com>
Subject: Re: Cluster auto activation design proposal
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary="089e0822159cbc46660555d99d60"
archived-at: Thu, 03 Aug 2017 13:47:11 -0000

--089e0822159cbc46660555d99d60
Content-Type: text/plain; charset="UTF-8"

I also would like to provide more use cases of how BLT is supposed to work
(let me call it this way until we come up with a better one):

   1. User creates new BLT using WebConsole or other tool and "applies" it
   to brand-new cluster.

   2. User starts up brand-new cluster with desired amount of nodes and
   activates it. At the moment of activation BLT is created with all server
   non-daemon nodes presented in the cluster.

   3. User starts up a cluster with previously prepared BLT -> when set of
   nodes in the cluster matches with BLT cluster gets automatically activated.

   4. User has an up-and-running active cluster and starts a few more
   nodes. They join the cluster but no partitions are assigned to them.
   User recreates BLT on new cluster topology -> partitions are assigned to
   new nodes.

   5. User takes out nodes from cluster (e.g. for maintenance purposes): no
   rebalance happens until user recreates BLT on new cluster topology.

   6. If some parameters reach critical levels (e.g. number of backups for
   a partition is too low) coordinator automatically recreates BLT and thus
   triggers rebalancing.


I hope these use cases will help to clarify purposes of the proposed
feature.

On Thu, Aug 3, 2017 at 4:08 PM, Alexey Goncharuk <alexey.goncharuk@gmail.com
> wrote:

> My understanding of Baseline Topology is the set of nodes which are
> *expected* to be in the cluster.
> Let me go a little bit further because BT (or whatever name we choose) may
> and will solve more issues than just auto-activation:
>
> 1) More graceful control over rebalancing than just rebalance delay. If a
> server is shut down for maintenance and there are enough backup nodes in
> the cluster, there is no need to rebalance.
> 2) Guarantee that there will be no conflicting key-value mappings due to
> incorrect cluster activation. For example, consider a scenario when there
> was a cluster of 10 nodes, then the cluster was shut down, started first 5
> nodes, activated, made some updates, shut down 5 nodes, start up other 5
> nodes, activate, make some updates, start up first 5 nodes. Currently,
> there is no way to determine that there was an incompatible topology change
> which leads to data inconsistency.
> 3) When a cluster is shutting down node-by-node, we must track a node which
> has 'seen' a partition last time and not activate the cluster until all
> nodes are present. Otherwise, again, we may activate too early and see
> outdated values.
>
> I do not want to add any 'faster' hacks here because they will only make
> the issue above appear more likely. Besides, BT should be available in 2.2
> anyway, so no need to rush with hacks.
>
> --AG
>
> 2017-08-03 15:09 GMT+03:00 Yakov Zhdanov <yzhdanov@apache.org>:
>
> > >Obvious connotation of "minimal set" is a set that cannot be decreased.
> >
> > >But lets consider the following case: user has a cluster of 50 nodes and
> > >decides to switch off 3 nodes for maintenance for a while. Ok, user just
> > >does it and then recreates this "minimal node set" to only 47 nodes.
> >
> > >So initial minimal node set was decreased - something counter-intuitive
> to
> > >me and may cause confusion as well.
> >
> > That was my point. If I have 50 nodes and 3 backups I can restart on 48,
> 49
> > and 50 without data loss. In case of 48 and 49 after cluster gets
> activated
> > missing backups are assigned and rebalancing starts.
> >
> > --Yakov
> >
>

--089e0822159cbc46660555d99d60--