ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: Persistence per memory policy configuration
Date Mon, 25 Sep 2017 23:22:46 GMT
Given that we already have a notion of CacheStore which comes from JCache
spec, I think having other stores may get confusing. I like
DurableMemoryConfiguration.

Other opinions?

D.

On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <vozerov@gridgain.com>
wrote:

> Dima, let's finalize the design first.
>
> As I understand, we are happy with idea to merge MemoryConfiguration
> and PersistentStoreConfiguration
> into something what I called DataConfiguration, and to rename
> MemoryPolicyConfiguration to DataRegionConfiguration.
>
> The only outstanding qurestion is whether DataConfiguration is a good name.
> I am not very happy with it, so let's think of other alternatives. Quick
> ideas:
> 1) StoreConfiguration - looks perfect to me - short and self-describing,
> but clashes a bit with existing CacheStore
> 2) DataStoreConfiguration - same as p.1, but the word "data" is not
> necessary IMO
> 3) PageStoreConfiguration? GIves a hint to our page-based architecture.
> 4) DurableMemoryConfiguration - aligns well with our docs, but I do not
> like it - too long and misleading
>
> Any other ideas?
>
> I would prefer to have either [StoreConfiguration +
> StoreRegionConfiguration] or [PageStoreConfiguration and
> PageStoreRegionConfiguration]. Looks clean and simple.
>
> Vladimir.
>
>
> On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <dsetrakyan@apache.org>
> wrote:
>
> > Vladimir,
> >
> > Can you please add the configuration example in the ticket?
> >
> > D.
> >
> > On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com> wrote:
> >
> > > Guys,
> > >
> > > I suggest we finalize the configuration changes in the original ticket
> > > then: https://issues.apache.org/jira/browse/IGNITE-6030 and proceed
> with
> > > the changes.
> > >
> > > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
> > >
> > > > Can we specify what metrics will look like? I think we should not
> just
> > > > blindly merge them.
> > > >
> > > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> > vozerov@gridgain.com>
> > > > wrote:
> > > >
> > > > > Denis,
> > > > >
> > > > > Makes sense. Thanks for catching it!
> > > > >
> > > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <dmagda@apache.org>
> > > wrote:
> > > > >
> > > > > > If we’re taking the consolidation path for Memory and Persistence
> > > > > > configurations then it makes sense to merge MemoryMetrics [1]
and
> > > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > > >
> > > > > > Agree?
> > > > > >
> > > > > > [1] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > apache/ignite/MemoryMetrics.html <https://ignite.apache.org/
> > > > > > releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html>
> > > > > > [2] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > apache/ignite/
> > > > > > PersistenceMetrics.html
> > > > > >
> > > > > > —
> > > > > > Denis
> > > > > >
> > > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > > > dsetrakyan@apache.org
> > > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > I like it.
> > > > > > >
> > > > > > > Alexey G, can you please chime in?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > > > > vozerov@gridgain.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Guys,
> > > > > > >>
> > > > > > >> Here is my proposal:
> > > > > > >>
> > > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > > *DataRegionConfiguration*.
> > > > > > >> 2) PersistenceConfiguration is merged with MemoryConfiguration
> > and
> > > > > > renamed
> > > > > > >> to ... *DataStorageConfiguration*! It has: common memory
> > settings
> > > > > (e.g.
> > > > > > >> default data region), persistence settings (e.g. WAL)
and a
> list
> > > of
> > > > > > >> DataRegionConfiguration beans.
> > > > > > >>
> > > > > > >> What we have in the end:
> > > > > > >>
> > > > > > >> <property name="dataConfiguration">
> > > > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > > > >>        <property name="pageSize" value="8192" />
> > > > > > >>        <property name="persistentStorePath" value="/my/path"
> />
> > > > > > >>        <property name="dataRegions">
> > > > > > >>            <list>
> > > > > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > > > > >>                    <property name="name" value="VOLATILE"
/>
> > > > > > >>                    <property name="maxSize"
> > value="1_000_000_000"
> > > />
> > > > > > >>                </bean>
> > > > > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > > > > >>                    <property name="name" value="PERSISTENT"
/>
> > > > > > >>                    <property name="maxSize"
> > value="1_000_000_000"
> > > />
> > > > > > >>                    <property name="persistent" value="true"
/>
> > > > > > >>                </bean>
> > > > > > >>            </list>
> > > > > > >>        </property>
> > > > > > >>    </bean>
> > > > > > >> </property>
> > > > > > >>
> > > > > > >>
> > > > > > >> Makes sense?
> > > > > > >>
> > > > > > >> Vladimir.
> > > > > > >>
> > > > > > >>
> > > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan
<
> > > > > > dsetrakyan@apache.org>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >>> Firstly all, why not call it DataPolicy instead
of
> > MemoryPolicy?
> > > > > > >> Secondly,
> > > > > > >>> why not set data policies directly on IgniteConfiguration.
> And
> > > > > lastly,
> > > > > > >> how
> > > > > > >>> about we combine memory and disk properties in
one bean with
> > > clear
> > > > > > naming
> > > > > > >>> convention?
> > > > > > >>>
> > > > > > >>> Here is the example. Note that all properties above
must
> start
> > > with
> > > > > > with
> > > > > > >>> "Memory" or "Disk".
> > > > > > >>>
> > > > > > >>> *IgniteConfiguration cfg = new IgniteConfiguration();*
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>>
> > > > > > >>>>
> > > > > > >>>>
> > > > > > >>>>
> > > > > > >>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > > > > > >>>> .setName("bla"),        .setMemoryMaxSize(1024),
// must be
> > > > greater
> > > > > > >> than
> > > > > > >>> 0,
> > > > > > >>>> since memory always needs to be enabled.
> > > > .setDiskMaxSize(0),
> > > > > //
> > > > > > >> if
> > > > > > >>>> greater than 0, then persistence is enabled.
   );*
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> I think this approach is much more concise and
straight
> > forward.
> > > > What
> > > > > > do
> > > > > > >>> you think?
> > > > > > >>>
> > > > > > >>> D.
> > > > > > >>>
> > > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov
<
> > > > > vozerov@gridgain.com
> > > > > > >
> > > > > > >>> wrote:
> > > > > > >>>
> > > > > > >>>> I prefer the second. Composition over inheritance
- this is
> > how
> > > > all
> > > > > > our
> > > > > > >>>> configuration is crafted.
> > > > > > >>>>
> > > > > > >>>> E.g. we do not have "CacheConfiguration" and
"
> > > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > > >>>> Instead, we have "CacheConfiguration.setCacheStoreFactory".
> > > > > > >>>>
> > > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk
<
> > > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > > >>>>
> > > > > > >>>>> Reiterating this based on some feedback
from PDS users.
> > > > > > >>>>>
> > > > > > >>>>> It might be confusing to configure persistence
with
> > > > "MemoryPolicy",
> > > > > > >> so
> > > > > > >>>>> another approach is to deprecate the old
names and
> introduce
> > a
> > > > new
> > > > > > >> name
> > > > > > >>>>> "DataRegion" because it reflects the actual
state when data
> > is
> > > > > stored
> > > > > > >>> on
> > > > > > >>>>> disk and partially in memory. I have two
options in mind,
> > each
> > > of
> > > > > > >> them
> > > > > > >>>>> looks acceptable to me, so I would like
to have some
> feedback
> > > > from
> > > > > > >> the
> > > > > > >>>>> community. Old configuration names will
be deprecated (but
> > > still
> > > > be
> > > > > > >>> taken
> > > > > > >>>>> if used for backward compatibility). Note,
that old names
> > > > > deprecation
> > > > > > >>>>> handles default configuration compatibility
very nicely -
> > > current
> > > > > PDS
> > > > > > >>>> users
> > > > > > >>>>> will not need to change anything to keep
everything
> working.
> > > The
> > > > > two
> > > > > > >>>>> options I mentioned are below:
> > > > > > >>>>>
> > > > > > >>>>> * we have two separate classes for in-memory
and persisted
> > data
> > > > > > >>> regions,
> > > > > > >>>>> so the configuration would look like so:
> > > > > > >>>>>
> > > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > > >>>>>
> > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > DataRegionsConfiguration()
> > > > > > >>>>>    .setDataRegions(
> > > > > > >>>>>        new MemoryDataRegion()
> > > > > > >>>>>            .setName("volatileCaches")
> > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > >>>>>        new PersistentDataRegion()
> > > > > > >>>>>            .setName("persistentCaches")
> > > > > > >>>>>            .setMaxMemorySize(...)
> > > > > > >>>>>            .setMaxDiskSize()));
> > > > > > >>>>>
> > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > >> PersistentStoreConfiguration()
> > > > > > >>> );
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >>>>> * we have one class for data region configuration,
but it
> > will
> > > > > have a
> > > > > > >>>>> sub-bean for persistence configuration:
> > > > > > >>>>>
> > > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > > >>>>>
> > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > DataRegionsConfiguration()
> > > > > > >>>>>    .setDataRegions(
> > > > > > >>>>>        new DataRegion()
> > > > > > >>>>>            .setName("volatileCaches")
> > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > >>>>>        new DataRegion()
> > > > > > >>>>>            .setName("persistentCaches")
> > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > >>>>>            .setPersistenceConfiguration(
> > > > > > >>>>>                new DataRegionPersistenceConfiguration()
> > > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > > >>>>>
> > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > >> PersistentStoreConfiguration()
> > > > > > >>> );
> > > > > > >>>>>
> > > > > > >>>>
> > > > > > >>>
> > > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message