ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: New API for changing configuration of persistent caches
Date Wed, 21 Nov 2018 15:22:36 GMT
Alex,

Spring XML is not easily available from API. Sometimes it simply doesn’t
exist.
Why not use builder instead?

Ignite.changeCache(“my_cache”,
Builder.create().setCacheMode(REPLICATED).build())

Advantages:
1) Available on all platforms
2) Expose only properties we support
3) Avoid race condition when configuration changes between configuration
read and method call (what could lead to a number of strange effects).

Makes sense?

ср, 21 нояб. 2018 г. в 18:16, Alexey Kuznetsov <akuznetsov@apache.org>:

> Vova, Ed
>
> I think in case when classes not available we can sent a string with cache
> XML configuration (bean) to server node that has all classes in classpath.
>
> What do you think?
>
> On Wed, Nov 21, 2018 at 10:10 PM Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
>
> > Eduard,
> >
> > Simple != correct. Let’s consider a simple use case: user want to change
> > PARTITIONED -> REPLICATED from .NET, but do not some classes from
> > CacheConfiguration. How do we solve this?
> >
> > Vladimir.
> >
> > ср, 21 нояб. 2018 г. в 18:02, Eduard Shangareev <
> > eduard.shangareev@gmail.com
> > >:
> >
> > > Vladimir,
> > >
> > > I propose not to change cache configuration in runtime but restart
> cache
> > > with the new compatible configuration on data which we have underfoot.
> > >
> > > What we could change:
> > > -backup count;
> > > -TRANSACTIONAL <-> ATOMIC;
> > > -REPLICATED - PARTITIONED;
> > > -other settings.
> > >
> > > So, yeah, it would be great to have a possibility to change some
> > properties
> > > in runtime. But right we don't any way to change anything except
> indexes
> > > and SQL fields.
> > >
> > > We already have all mechanism to do this.
> > > The main issue is to make it reliable and exclude cases when we could
> > come
> > > to the unrecoverable state.
> > >
> > > So, I suggest keeping the solution as simple as possible.
> > > For indexes clashes and ClassNotFoundException we could revert
> > > configuration update and start with the old configuration.
> > >
> > >
> > > On Wed, Nov 21, 2018 at 5:44 PM Vladimir Ozerov <vozerov@gridgain.com>
> > > wrote:
> > >
> > > > Eduard,
> > > >
> > > > Got it. Please take the following things in count during design:
> > > >
> > > > 1) Two distinct PMEs might not work well. Consider a situation w1hen
> I
> > > > decided to move a cache with index "MY_INDEX" from schema A to schema
> > B.
> > > > While cache was stopped, another cache with index "MY_INDEX" was
> > created
> > > in
> > > > schema B. Now first cache cannot start due to index name conflict.
> > > > 2) Cancelling index creation is also bad idea because this is
> > potentially
> > > > long operation. Instead, most likely that we should wait to
> concurrent
> > > > schema operations to finish first. That is, all operations on cache
> > > should
> > > > be ordered wrt each other somehow
> > > > 3) Why do we think that cache restart will be needed at all? We have
> a
> > > lot
> > > > of configuration properties which could be changed safely either
> > without
> > > > PME or with a single PME. - rebalance properties, cache store
> > properties
> > > > (especially write-behind stuff), some query properties (e.g. "detail
> > > > metrics"), etc.. In essence, it seems that >50% of properties could
> be
> > > > changed without cache restart, other 25% will not be supported, and
> the
> > > > rest may require restart.
> > > > 4) Client nodes and thin client may not have necessary classes in
> > > > classpath. E.g. consider a user which want to change rebalance
> timeout
> > > for
> > > > cache, but do not have configured interceptor in classpath. With
> > proposed
> > > > API it will be impossible. This is especially true for non-Java
> > clients.
> > > >
> > > > That said, I think we should consider another API which will not
> > require
> > > > full CacheConfiguration object. This might be a kind of builder or
> so.
> > > And
> > > > once user set properties he want to change to the builder, we can
> > analyze
> > > > them and either change them in runtime without PME, change with a
> > single
> > > > PME or change with full cache restart.
> > > >
> > > > What do you think?
> > > >
> > > > Vladimir.
> > > >
> > > > On Wed, Nov 21, 2018 at 5:01 PM Eduard Shangareev <
> > > > eshangareev@gridgain.com>
> > > > wrote:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > 1) Affinity could be changed, but count of partition couldn't be.
> > > > > 2) So it would trigger 2 PME. Dynamic start and stop.
> > > > > 3) In theory, should cancel them and new setting should be applied.
> > How
> > > > it
> > > > > works now? Create an index and stop node, for example.
> > > > >
> > > > > On Wed, Nov 21, 2018 at 4:56 PM Vladimir Ozerov <
> > vozerov@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Ed,
> > > > > >
> > > > > > Several questions from my side:
> > > > > > 1) If we do not allow to change the most demanded by users things
> > > like
> > > > > > affinity or persistence/in-memory, then what kind of
> configuration
> > > > > > properties do we expect to be changed? Can we have several
> > examples?
> > > > > > 2) How will it interact with PME?
> > > > > > 3) How will it interact with CREATE INDEX and ALTER TABLE
> commands?
> > > > > >
> > > > > > On Wed, Nov 21, 2018 at 4:48 PM Eduard Shangareev <
> > > > > > eduard.shangareev@gmail.com> wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I propose new public API to change the cache configuration
of
> > > > > persistent
> > > > > > > caches with keeping data.
> > > > > > >
> > > > > > > It would look like:
> > > > > > >
> > > > > > > Ignite ignite = ...;
> > > > > > > ignite.restartCaches(cfg1, ... cfgN);
> > > > > > >
> > > > > > > where cfgX is a new cache configuration, which contains
the
> name
> > of
> > > > an
> > > > > > > existing persistent cache.
> > > > > > >
> > > > > > > The obvious limitation:
> > > > > > > - affinity key mapping couldn't be changed;
> > > > > > > - count of partitions couldn't be changed;
> > > > > > > - MVCC couldn't be turned off/on;
> > > > > > > - persistent couldn't be turned off;
> > > > > > > - group settings couldn't be changed (group name);
> > > > > > > - if cache belongs to group it's needed to restart all
of them.
> > > > > > >
> > > > > > > Failure scenario is the crucial thing (and most difficult):
> > > > > > > - initiator fail;
> > > > > > > - cluster restart at any stage;
> > > > > > > - joining/starting offline nodes.
> > > > > > >
> > > > > > > Some thoughts about implementation:
> > > > > > > - stop cache with destroy=false;
> > > > > > > - start cache dynamically with new configuration;
> > > > > > > - if indexes settings changed - remove index.bin to start
> > > indexation;
> > > > > > > - change blt-history when start cache initiated to not
allow
> join
> > > > nodes
> > > > > > > with old configuration;
> > > > > > > - use restartId (IGNITE-8911) to not allow to start cache
in
> > > between.
> > > > > > >
> > > > > > > Your thoughts? Would it be a useful feature?
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Eduard.
> > > > >
> > > >
> > >
> >
>
>
> --
> Alexey Kuznetsov
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message