Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <CA+0=VoVFnnXfaLE_MiFzv0GvErwnaWtCaCtGh3KaFu3vAUARzQ@mail.gmail.com>
References: <CALH+G9oEmZ0J1B2aCquKb4HMGKjoh_84OwMnPF1=ZtaFhfsWYg@mail.gmail.com>
 <CAHKVN8jMAKeDtLFUjP-VDpYF4H4gZntuH4z-jkkvZ4DyH5wBkg@mail.gmail.com>
 <3B2BD029-1490-4A8C-94F5-EF54FB291E99@apache.org> <CABDss3iAYrDhgOPpPHe4JYu=yz5FnCftKv09myn81F+2DwSXiQ@mail.gmail.com>
 <CA+0=VoVZkWQFu2Wv+PkLWbbvUC1cqozm4puRxbHzOX2C95_FKw@mail.gmail.com>
 <CAHKVN8iqrKmK=kWx4ZSXqFEC6tbXjckBarffR8kj0EmV4ujuJA@mail.gmail.com>
 <CAHKVN8joz-nNgjeXe_faw_E5u30z34kZRS7gGQouyQo6iG7_Fw@mail.gmail.com>
 <CA+0=VoW2kwJG7rVf6mxjjdnBPb-o8zyFAbPNtxUkCgbtQpK83g@mail.gmail.com>
 <CAHKVN8iwzTdhph5ijReG-nSjA_31q8Ld72bio+YDxwR=18KX6w@mail.gmail.com>
 <CA+0=VoUkREqbn4KbGHNxh68hS_QQoKgrfXzJbs7Qiz_+X0w2Ew@mail.gmail.com>
 <1503076731467-21066.post@n4.nabble.com> <CA+0=VoWBan6MFH_g1bzwP2FGDBMOn4Bo1hYObdrz6dqQfTCpnQ@mail.gmail.com>
 <CAHKVN8hVbGkwPqT_nbUq-zku_fQ3j9OoETEKJg9NBmw-pccV-A@mail.gmail.com>
 <CA+0=VoUcX+3+HgY+iEC1uGLq+OmABD+H60wSXYzoFvFVfe_sQA@mail.gmail.com>
 <CAHKVN8ha6wLkGy-++jsTrYxe1g-S-V=zN3TiHW1n5EyKXGcXaA@mail.gmail.com>
 <CA+0=VoUK1i3UfuB9+DJ8HTodVuOO0Tw3jMdJJWM8cSf4QpbhFQ@mail.gmail.com>
 <CAHKVN8g3XCuhWm32_bdynXVzARh2t2+PQSRRGf3X6+bFcWZLWA@mail.gmail.com>
 <CACpHDb=t-OLq9FeaFVsTJG6pxPUz1Nc4g5DmJ5ijvtP331ZD9A@mail.gmail.com>
 <CAHKVN8idrF6dKsiS54XCekKp3MqwSWpr+6RojDq4Q-fdi1T3=Q@mail.gmail.com>
 <CA+0=VoU7kbYMdAWac_J8yASRVPvEkjX2G-904sm1kMxcZWEXbg@mail.gmail.com>
 <CAHKVN8g8vwZCeaDJppDPNcvRfjOaK+jb2k2b7UXVUTMmH5C+5w@mail.gmail.com> <CA+0=VoVFnnXfaLE_MiFzv0GvErwnaWtCaCtGh3KaFu3vAUARzQ@mail.gmail.com>
From: Sergey Chugunov <sergey.chugunov@gmail.com>
Date: Mon, 4 Sep 2017 16:13:01 +0300
Message-ID: <CAHKVN8hy+DfRipvng5nwn_wdNoAY=OeX=Yzyc8LGkX+NSZka+g@mail.gmail.com>
Subject: Re: Cluster auto activation design proposal
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary="f403045c19b239ee7005585ce166"
archived-at: Mon, 04 Sep 2017 13:13:56 -0000

--f403045c19b239ee7005585ce166
Content-Type: text/plain; charset="UTF-8"

Dmitriy,

I like the idea of ClusterActivator interface.

From user perspective it provides the same functionality as the setter but
in more clear and intuitive way.

Also it gives us a good place to put all the documentation about the
feature.

Any other opinions?

On Fri, Sep 1, 2017 at 2:35 PM, Dmitriy Setrakyan <dsetrakyan@apache.org>
wrote:

> How about this:
>
>
> > *interface ClusterActivator {*
> > *    boolean activate(Collection<IgniteNode> nodes);**}*
>
>
> Out of the box, we can provide this implementation of the activation
> filter:
>
>
> >
> > *ClusterInitialActiveSet implements ClusterActivator { *
> > *    InigeInitialActiveSet(String... addresses);**}*
>
>
> Then user configuration can look as follows:
>
> *IgniteConfiguration.setActivationFilter(new
> > ClusterInitialActiveSet("1.2.3.4", "4.3.2.1", etc));*
>
>
> Thoughts?
>
> D.
>
> On Fri, Sep 1, 2017 at 1:47 PM, Sergey Chugunov <sergey.chugunov@gmail.com
> >
> wrote:
>
> > Dmitriy,
> >
> > The idea is interesting however I cannot come up with a clear use case
> > which can be widely adopted.
> > I would give users a simple API at first to cover 80% of their needs and
> > then collect some feedback and start thinking about adding new
> > functionality.
> >
> > Makes sense?
> >
> > Sergey.
> >
> > On Thu, Aug 31, 2017 at 3:55 AM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> >
> > > Hm... Can we also ask user to optionally provide a predicate which will
> > > receive a collection of nodes started so far and return true if the
> > > activation should happen? Will it be useful?
> > >
> > > On Wed, Aug 30, 2017 at 6:28 PM, Sergey Chugunov <
> > > sergey.chugunov@gmail.com>
> > > wrote:
> > >
> > > > Nick,
> > > >
> > > > As I summed up in this thread above, calling setter for initial
> > > activation
> > > > nodes is not the only option:
> > > >
> > > >    1. user starts up new cluster of desired number of nodes and
> > activates
> > > >    it using existing API.
> > > >    BLT is created with all nodes presented in the cluster at the
> moment
> > > of
> > > >    activation, no API is needed;
> > > >
> > > >    2. user prepares BLT using web-console or visor CMD tools and sets
> > it
> > > to
> > > >    the cluster. New API setter is needed:
> > > >    Ignite.activation().setInitialActivationNodes(Collection<
> > ClusterNode>
> > > >    nodes);
> > > >
> > > >    3. user provides via static configuration a list of nodes that are
> > > >    expected to be in the cluster.
> > > >    User starts nodes one by one; when all preconfigured nodes are
> > started
> > > >    cluster is activated and BLT is created.
> > > >    As list of nodes may be huge it is provided via separate file to
> > avoid
> > > >    flooding main configuration.
> > > >
> > > > So the option you proposed is already in the list.
> > > >
> > > > As for idea of activating cluster based only on number of nodes may
> be
> > > > risky.
> > > > E.g. if user starts up with data stored on disk and unexpected node
> > joins
> > > > the topology.
> > > > Cluster will get activated with N-1 nodes where all the data is
> > presented
> > > > and one node completely empty. Data loss may happen in such scenario.
> > > >
> > > > Thanks,
> > > > Sergey.
> > > >
> > > >
> > > > On Wed, Aug 30, 2017 at 4:23 PM, Nick Pordash <nickpordash@gmail.com
> >
> > > > wrote:
> > > >
> > > > > How is a user expected to produce a collection of ClusterNode prior
> > to
> > > > all
> > > > > of the expected nodes joining? Users don't create instances of
> this,
> > so
> > > > as
> > > > > far as I can tell it would have to be retrieved from IgniteCluster.
> > > > > However, would doing that and calling the proposed method be really
> > any
> > > > > different than calling Ignite.activate and using the current set of
> > > > server
> > > > > nodes as that collection?
> > > > >
> > > > > From a user's perspective is it really necessary that specific
> nodes
> > > need
> > > > > to be identified vs saying that they expect N server nodes to be in
> > the
> > > > > cluster for auto activation?
> > > > >
> > > > > -Nick
> > > > >
> > > > > On Wed, Aug 30, 2017, 1:23 AM Sergey Chugunov <
> > > sergey.chugunov@gmail.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > Now I see your point and I think you're right.
> > > > > >
> > > > > > We can give end-user a simple setter like
> > > > > > Ignite::activation::setInitialActivationNodes(
> > > Collection<ClusterNode>
> > > > > > nodes) to provide collection of nodes that grid must reach to
> > > activate
> > > > > > automatically.
> > > > > >
> > > > > > And then using the collection we'll create BaselineTopology
> > > abstraction
> > > > > > internally.
> > > > > >
> > > > > > As a result user won't be exposed to our internal abstractions
> and
> > > will
> > > > > > work with intuitive concept of collection of nodes.
> > > > > >
> > > > > > Thanks,
> > > > > > Sergey.
> > > > > >
> > > > > >
> > > > > > On Mon, Aug 28, 2017 at 4:39 PM, Dmitriy Setrakyan <
> > > > > dsetrakyan@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Sergey, the interface you are suggesting is internal, not
> > external.
> > > > Why
> > > > > > > should user ever see it or care about it?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Mon, Aug 28, 2017 at 3:18 PM, Sergey Chugunov <
> > > > > > > sergey.chugunov@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Dmitriy,
> > > > > > > >
> > > > > > > > It was my misunderstanding, I believe that setter is not
> enough
> > > and
> > > > > we
> > > > > > > need
> > > > > > > > a full-fledged entity.
> > > > > > > >
> > > > > > > > We should also be able to check if BLTs are compatible.
> > Interface
> > > > > looks
> > > > > > > > like this and use case for this functionality is described
> > below.
> > > > > > > >
> > > > > > > > interface BaselineTopology {
> > > > > > > >    Collection<ClusterNode> nodes();
> > > > > > > >    boolean isCompatibleWith(BaselineTopology blt);
> > > > > > > > }
> > > > > > > >
> > > > > > > > Let's consider the following scenario:
> > > > > > > >
> > > > > > > >    1. We have a grid with N nodes: it is up, active and has
> > data
> > > in
> > > > > it.
> > > > > > > ->
> > > > > > > >    BLT #1 created.
> > > > > > > >    2. We shutdown the grid. Then divide it into two parts:
> > > > Part1_grid
> > > > > > and
> > > > > > > >    Part2_grid.
> > > > > > > >    3. We start and activate Part1_grid . Topology has changed
> > ->
> > > > > BLT#2
> > > > > > > >    created.
> > > > > > > >    After that we shutdown that Part1_grid.
> > > > > > > >    4. We start and activate Part2_grid. Topology also has
> > changed
> > > > ->
> > > > > > > BLT#3
> > > > > > > >    created.
> > > > > > > >    5. Then we start Part1_grid and it's nodes try to join
> > > > Part2_grid.
> > > > > > > >
> > > > > > > >
> > > > > > > > If join is successful we have an undefined state of the
> > resulting
> > > > > grid:
> > > > > > > > values for the same key may (and will) differ between grid
> > parts.
> > > > > > > >
> > > > > > > > So to prevent this we should keep nodes with BLT#2 from
> joining
> > > the
> > > > > > grid
> > > > > > > > with BLT#3. And we should fail nodes with an error message.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Sergey.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Aug 23, 2017 at 5:47 AM, Dmitriy Setrakyan <
> > > > > > > dsetrakyan@apache.org>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Sergey, I am still confused. What is the BaselineTopology
> > > > interface
> > > > > > in
> > > > > > > > your
> > > > > > > > > example? I thought that you agreed with me that we simply
> > need
> > > a
> > > > > > setter
> > > > > > > > for
> > > > > > > > > activation nodes, no?
> > > > > > > > >
> > > > > > > > > D.
> > > > > > > > >
> > > > > > > > > On Tue, Aug 22, 2017 at 4:47 AM, Sergey Chugunov <
> > > > > > > > > sergey.chugunov@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Dmitriy,
> > > > > > > > > >
> > > > > > > > > > As I understand you use the term "minimalActivationNodes"
> > as
> > > a
> > > > > > > synonym
> > > > > > > > > for
> > > > > > > > > > BaselineTopology concept.
> > > > > > > > > > In that case I agree with you that we can replace both
> > > > > "establish*"
> > > > > > > > > methods
> > > > > > > > > > with a simple setter method (see below in summary).
> > > > > > > > > >
> > > > > > > > > > Summing up the whole discussion I see the functionality
> as
> > > > > > following:
> > > > > > > > > >
> > > > > > > > > > New concept BaselineTopology is introduced. The main
> > features
> > > > it
> > > > > > > > enables
> > > > > > > > > > are:
> > > > > > > > > >
> > > > > > > > > >    1. automatic activation of cluster;
> > > > > > > > > >
> > > > > > > > > >    2. easy management of cluster topology changes
> (planned
> > > > nodes
> > > > > > > > > >    maintenance, adding new nodes etc);
> > > > > > > > > >
> > > > > > > > > >    3. eliminating of rebalancing traffic on short-term
> node
> > > > > > failures.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Use cases to create BLT:
> > > > > > > > > >
> > > > > > > > > >    1. user starts up new cluster of desired number of
> nodes
> > > and
> > > > > > > > activates
> > > > > > > > > >    it using existing API. BLT is created with all nodes
> > > > presented
> > > > > > in
> > > > > > > > the
> > > > > > > > > >    cluster at the moment of activation, no API is needed;
> > > > > > > > > >
> > > > > > > > > >    2. user prepares BLT using web-console or visor CMD
> > tools
> > > > and
> > > > > > sets
> > > > > > > > it
> > > > > > > > > to
> > > > > > > > > >    the cluster. New API setter is needed:
> > > > > > > > > >    Ignite.activation().setBaselin
> > eTopology(BaselineTopology
> > > > > blt);
> > > > > > > > > >
> > > > > > > > > >    3. user provides via static configuration a list of
> > nodes
> > > > that
> > > > > > are
> > > > > > > > > >    expected to be in the cluster.
> > > > > > > > > >    User starts nodes one by one; when all preconfigured
> > nodes
> > > > are
> > > > > > > > started
> > > > > > > > > >    cluster is activated and BLT is created.
> > > > > > > > > >    As list of nodes may be huge it is provided via
> separate
> > > > file
> > > > > to
> > > > > > > > avoid
> > > > > > > > > >    flooding main configuration.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Igniters, does this description match with your
> > understanding
> > > > of
> > > > > > > > > > functionality? If it does I'll create a set of tickets
> and
> > > > start
> > > > > > > > working
> > > > > > > > > on
> > > > > > > > > > implementation.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Sergey.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Sat, Aug 19, 2017 at 5:41 PM, Dmitriy Setrakyan <
> > > > > > > > > dsetrakyan@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I still do not see why anyone would explicitly call
> > these 2
> > > > > > > methods:
> > > > > > > > > > >
> > > > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > > > BaselineTopology
> > > > > > > > > > bltTop);*
> > > > > > > > > > >
> > > > > > > > > > > For example, if a web console, or some other admin
> > process,
> > > > > want
> > > > > > to
> > > > > > > > > > > automatically set currently started nodes as the
> baseline
> > > > > > topology,
> > > > > > > > > > > shouldn't they just call a setter for
> > > minimalActivationNodes?
> > > > > > > > > > >
> > > > > > > > > > > D.
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Aug 18, 2017 at 10:18 AM, Alexey Dmitriev <
> > > > > > > > > > admitriev@gridgain.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > API is proposed in the head of the thread by Sergey,
> > as I
> > > > > > > > understood:
> > > > > > > > > > > > ______________________________
> ________________________
> > > > > > > > > > > >
> > > > > > > > > > > > API for BaselineTopology manipulation may look like
> > this:
> > > > > > > > > > > >
> > > > > > > > > > > > *Ignite::activation::establishBaselineTopology();*
> > > > > > > > > > > > *Ignite::activation::establishBaselineTopology(
> > > > > BaselineTopology
> > > > > > > > > > > bltTop);*
> > > > > > > > > > > >
> > > > > > > > > > > > Both methods will establish BT and activate cluster
> > once
> > > it
> > > > > is
> > > > > > > > > > > established.
> > > > > > > > > > > >
> > > > > > > > > > > > The first one allows user to establish BT using
> current
> > > > > > topology.
> > > > > > > > If
> > > > > > > > > > any
> > > > > > > > > > > > changes happen to the topology during establishing
> > > process,
> > > > > > user
> > > > > > > > will
> > > > > > > > > > be
> > > > > > > > > > > > notified and allowed to proceed or abort the
> procedure.
> > > > > > > > > > > >
> > > > > > > > > > > > Second method allows to use some
> > monitoring'n'management
> > > > > tools
> > > > > > > like
> > > > > > > > > > > > WebConsole where user can prepare a list of nodes,
> > using
> > > > them
> > > > > > > > create
> > > > > > > > > a
> > > > > > > > > > BT
> > > > > > > > > > > > and send to the cluster a command to finally
> establish
> > > it.
> > > > > > > > > > > >
> > > > > > > > > > > > From high level BaselineTopology entity contains only
> > > > > > collection
> > > > > > > of
> > > > > > > > > > > nodes:
> > > > > > > > > > > >
> > > > > > > > > > > > *BaselineTopology {*
> > > > > > > > > > > > *  Collection<TopologyNode> nodes;*
> > > > > > > > > > > > *}*
> > > > > > > > > > > >
> > > > > > > > > > > > *TopologyNode* here contains information about node -
> > its
> > > > > > > > consistent
> > > > > > > > > id
> > > > > > > > > > > and
> > > > > > > > > > > > set of user attributes used to calculate affinity
> > > function.
> > > > > > > > > > > > ____________________________________________
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > View this message in context: http://apache-ignite-
> > > > > > > > > > > > developers.2346864.n4.nabble.
> > > com/Cluster-auto-activation-
> > > > > > > > > > > > design-proposal-tp20295p21066.html
> > > > > > > > > > > > Sent from the Apache Ignite Developers mailing list
> > > archive
> > > > > at
> > > > > > > > > > > Nabble.com.
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

--f403045c19b239ee7005585ce166--