ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eduard Shangareev <eduard.shangar...@gmail.com>
Subject Re: New API for changing configuration of persistent caches
Date Wed, 21 Nov 2018 15:14:09 GMT
Vladimir,

How does .Net user start caches right now?

On Wed, Nov 21, 2018 at 6:10 PM Vladimir Ozerov <vozerov@gridgain.com>
wrote:

> Eduard,
>
> Simple != correct. Let’s consider a simple use case: user want to change
> PARTITIONED -> REPLICATED from .NET, but do not some classes from
> CacheConfiguration. How do we solve this?
>
> Vladimir.
>
> ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> eduard.shangareev@gmail.com
> >:
>
> > Vladimir,
> >
> > I propose not to change cache configuration in runtime but restart cache
> > with the new compatible configuration on data which we have underfoot.
> >
> > What we could change:
> > -backup count;
> > -TRANSACTIONAL <-> ATOMIC;
> > -REPLICATED - PARTITIONED;
> > -other settings.
> >
> > So, yeah, it would be great to have a possibility to change some
> properties
> > in runtime. But right we don't any way to change anything except indexes
> > and SQL fields.
> >
> > We already have all mechanism to do this.
> > The main issue is to make it reliable and exclude cases when we could
> come
> > to the unrecoverable state.
> >
> > So, I suggest keeping the solution as simple as possible.
> > For indexes clashes and ClassNotFoundException we could revert
> > configuration update and start with the old configuration.
> >
> >
> > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <vozerov@gridgain.com>
> > wrote:
> >
> > > Eduard,
> > >
> > > Got it. Please take the following things in count during design:
> > >
> > > 1) Two distinct PMEs might not work well. Consider a situation w1hen I
> > > decided to move a cache with index "MY_INDEX" from schema A to schema
> B.
> > > While cache was stopped, another cache with index "MY_INDEX" was
> created
> > in
> > > schema B. Now first cache cannot start due to index name conflict.
> > > 2) Cancelling index creation is also bad idea because this is
> potentially
> > > long operation. Instead, most likely that we should wait to concurrent
> > > schema operations to finish first. That is, all operations on cache
> > should
> > > be ordered wrt each other somehow
> > > 3) Why do we think that cache restart will be needed at all? We have a
> > lot
> > > of configuration properties which could be changed safely either
> without
> > > PME or with a single PME. - rebalance properties, cache store
> properties
> > > (especially write-behind stuff), some query properties (e.g. "detail
> > > metrics"), etc.. In essence, it seems that >50% of properties could be
> > > changed without cache restart, other 25% will not be supported, and the
> > > rest may require restart.
> > > 4) Client nodes and thin client may not have necessary classes in
> > > classpath. E.g. consider a user which want to change rebalance timeout
> > for
> > > cache, but do not have configured interceptor in classpath. With
> proposed
> > > API it will be impossible. This is especially true for non-Java
> clients.
> > >
> > > That said, I think we should consider another API which will not
> require
> > > full CacheConfiguration object. This might be a kind of builder or so.
> > And
> > > once user set properties he want to change to the builder, we can
> analyze
> > > them and either change them in runtime without PME, change with a
> single
> > > PME or change with full cache restart.
> > >
> > > What do you think?
> > >
> > > Vladimir.
> > >
> > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > eshangareev@gridgain.com>
> > > wrote:
> > >
> > > > Vladimir,
> > > >
> > > > 1) Affinity could be changed, but count of partition couldn't be.
> > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > 3) In theory, should cancel them and new setting should be applied.
> How
> > > it
> > > > works now? Create an index and stop node, for example.
> > > >
> > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> vozerov@gridgain.com>
> > > > wrote:
> > > >
> > > > > Hi Ed,
> > > > >
> > > > > Several questions from my side:
> > > > > 1) If we do not allow to change the most demanded by users things
> > like
> > > > > affinity or persistence/in-memory, then what kind of configuration
> > > > > properties do we expect to be changed? Can we have several
> examples?
> > > > > 2) How will it interact with PME?
> > > > > 3) How will it interact with CREATE INDEX and ALTER TABLE commands?
> > > > >
> > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > eduard.shangareev@gmail.com> wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I propose new public API to change the cache configuration of
> > > > persistent
> > > > > > caches with keeping data.
> > > > > >
> > > > > > It would look like:
> > > > > >
> > > > > > Ignite ignite = ...;
> > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > >
> > > > > > where cfgX is a new cache configuration, which contains the
name
> of
> > > an
> > > > > > existing persistent cache.
> > > > > >
> > > > > > The obvious limitation:
> > > > > > - affinity key mapping couldn't be changed;
> > > > > > - count of partitions couldn't be changed;
> > > > > > - MVCC couldn't be turned off/on;
> > > > > > - persistent couldn't be turned off;
> > > > > > - group settings couldn't be changed (group name);
> > > > > > - if cache belongs to group it's needed to restart all of them.
> > > > > >
> > > > > > Failure scenario is the crucial thing (and most difficult):
> > > > > > - initiator fail;
> > > > > > - cluster restart at any stage;
> > > > > > - joining/starting offline nodes.
> > > > > >
> > > > > > Some thoughts about implementation:
> > > > > > - stop cache with destroy=false;
> > > > > > - start cache dynamically with new configuration;
> > > > > > - if indexes settings changed - remove index.bin to start
> > indexation;
> > > > > > - change blt-history when start cache initiated to not allow
join
> > > nodes
> > > > > > with old configuration;
> > > > > > - use restartId (IGNITE-8911) to not allow to start cache in
> > between.
> > > > > >
> > > > > > Your thoughts? Would it be a useful feature?
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Eduard.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message