Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <CA+0=VoXYMsGKSEUt2eQdrZaqoUjCETENOYME4Otov9oHBDa=4w@mail.gmail.com>
References: <CABDss3idR5Y6dgDHr7aKaVUkrOmJn391Q9KE6-hyNw3oVQtsSA@mail.gmail.com>
 <CAGt6o8kCA-Q6_2g_L4YYJZ3tvUWM2qCmvUAqiYZ8G6xasmKE1A@mail.gmail.com>
 <CAK2_iMPZoc0v4vJW4+i06s3OhtJO7JMnhFwwC6ENXgJuxLWZaQ@mail.gmail.com>
 <CABDss3j_2YDNxu-9iy38NT=7_D84kP3AYTw60Urfnzmx39SjCQ@mail.gmail.com>
 <CAGt6o8=-ug99JCDhEwc+yG0pLzwpJxCb_jJ_Z0k3_1=a0Ov-Bw@mail.gmail.com>
 <CABDss3jxNeypT19b7Nr9TrQE9N_TibJXk9gk-R9wDSmFb3ZcWg@mail.gmail.com>
 <CA+0=VoV7To7f4XoTo8WiQr3M2r82Sz_PTYjT=357=fcEtFO-2A@mail.gmail.com>
 <6b03926e-35c5-9229-d9d2-6f2f74b0e1a7@ya.ru> <CAGcMBHgXR-FHr9nAykZ7ERYfrpJd9a8E2mqhb0+jY9_rHu0naA@mail.gmail.com>
 <CABDss3haMQ+AJq_eXUkcb_PCorzMHiwRmqpEoqSZ5dq-hfU2HQ@mail.gmail.com>
 <CABDss3gaNAP5mVKmMLK=_wQgkGHqdBu9uujFjscO8Fbrna+thA@mail.gmail.com>
 <CAGt6o8kLSjkTSqPNfQpAjqNKNAVYzU+LxuBMePwiAjJh6GDFTQ@mail.gmail.com>
 <CA+0=VoX5a1tM15Ouzxhj5bsKVwqZoFDOhuiN=2runieU621bSg@mail.gmail.com>
 <CAGt6o8mLzdoE+YkNJa2bVZz5KV2=bNBZKFXo9DUdpBgJ=btvvQ@mail.gmail.com>
 <CA+0=VoWTh5qod=hTSkssVsFAre6bAnWp4gZUqTd7gxG8O9YzTQ@mail.gmail.com>
 <2B2EE969-7B3C-4772-8F95-BDF9DCD46B39@apache.org> <CAGt6o8=O7q+Tr2CbhjpxjsUPyz5M2fFzJak9xOHZs+Jgxs29eQ@mail.gmail.com>
 <CA+0=VoXOhRGwTy9cC=PMwx6-qOqd7-jmOo2pKvNB4kCYcD1dTw@mail.gmail.com>
 <CABDss3gyR=in5+TnS6vFboHw1JPay+MQDezek9TMjT5Kyzaz7g@mail.gmail.com> <CA+0=VoXYMsGKSEUt2eQdrZaqoUjCETENOYME4Otov9oHBDa=4w@mail.gmail.com>
From: Vladimir Ozerov <vozerov@gridgain.com>
Date: Mon, 25 Sep 2017 22:24:47 +0300
Message-ID: <CAGt6o8k=nLNeNAkNqrBRsACrsd7GeUHo+5LXvNAiNgKdZqPung@mail.gmail.com>
Subject: Re: Persistence per memory policy configuration
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary="94eb2c122f06072f55055a0883e6"
archived-at: Mon, 25 Sep 2017 19:24:56 -0000

--94eb2c122f06072f55055a0883e6
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Dima, let's finalize the design first.

As I understand, we are happy with idea to merge MemoryConfiguration
and PersistentStoreConfiguration
into something what I called DataConfiguration, and to rename
MemoryPolicyConfiguration to DataRegionConfiguration.

The only outstanding qurestion is whether DataConfiguration is a good name.
I am not very happy with it, so let's think of other alternatives. Quick
ideas:
1) StoreConfiguration - looks perfect to me - short and self-describing,
but clashes a bit with existing CacheStore
2) DataStoreConfiguration - same as p.1, but the word "data" is not
necessary IMO
3) PageStoreConfiguration? GIves a hint to our page-based architecture.
4) DurableMemoryConfiguration - aligns well with our docs, but I do not
like it - too long and misleading

Any other ideas?

I would prefer to have either [StoreConfiguration +
StoreRegionConfiguration] or [PageStoreConfiguration and
PageStoreRegionConfiguration]. Looks clean and simple.

Vladimir.


On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <dsetrakyan@apache.org>
wrote:

> Vladimir,
>
> Can you please add the configuration example in the ticket?
>
> D.
>
> On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
> > Guys,
> >
> > I suggest we finalize the configuration changes in the original ticket
> > then: https://issues.apache.org/jira/browse/IGNITE-6030 and proceed wit=
h
> > the changes.
> >
> > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
> >
> > > Can we specify what metrics will look like? I think we should not jus=
t
> > > blindly merge them.
> > >
> > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> vozerov@gridgain.com>
> > > wrote:
> > >
> > > > Denis,
> > > >
> > > > Makes sense. Thanks for catching it!
> > > >
> > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <dmagda@apache.org>
> > wrote:
> > > >
> > > > > If we=E2=80=99re taking the consolidation path for Memory and Per=
sistence
> > > > > configurations then it makes sense to merge MemoryMetrics [1] and
> > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > >
> > > > > Agree?
> > > > >
> > > > > [1] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > apache/ignite/MemoryMetrics.html <https://ignite.apache.org/
> > > > > releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html>
> > > > > [2] https://ignite.apache.org/releases/latest/javadoc/org/
> > > apache/ignite/
> > > > > PersistenceMetrics.html
> > > > >
> > > > > =E2=80=94
> > > > > Denis
> > > > >
> > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > > dsetrakyan@apache.org
> > > > >
> > > > > wrote:
> > > > > >
> > > > > > I like it.
> > > > > >
> > > > > > Alexey G, can you please chime in?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > > > vozerov@gridgain.com>
> > > > > > wrote:
> > > > > >
> > > > > >> Guys,
> > > > > >>
> > > > > >> Here is my proposal:
> > > > > >>
> > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > *DataRegionConfiguration*.
> > > > > >> 2) PersistenceConfiguration is merged with MemoryConfiguration
> and
> > > > > renamed
> > > > > >> to ... *DataStorageConfiguration*! It has: common memory
> settings
> > > > (e.g.
> > > > > >> default data region), persistence settings (e.g. WAL) and a li=
st
> > of
> > > > > >> DataRegionConfiguration beans.
> > > > > >>
> > > > > >> What we have in the end:
> > > > > >>
> > > > > >> <property name=3D"dataConfiguration">
> > > > > >>    <bean class=3D"o.a.i.DataConfiguration">
> > > > > >>        <property name=3D"pageSize" value=3D"8192" />
> > > > > >>        <property name=3D"persistentStorePath" value=3D"/my/pat=
h" />
> > > > > >>        <property name=3D"dataRegions">
> > > > > >>            <list>
> > > > > >>                <bean class=3D"o.a.i.DataRegionConfiguration">
> > > > > >>                    <property name=3D"name" value=3D"VOLATILE" =
/>
> > > > > >>                    <property name=3D"maxSize"
> value=3D"1_000_000_000"
> > />
> > > > > >>                </bean>
> > > > > >>                <bean class=3D"o.a.i.DataRegionConfiguration">
> > > > > >>                    <property name=3D"name" value=3D"PERSISTENT=
" />
> > > > > >>                    <property name=3D"maxSize"
> value=3D"1_000_000_000"
> > />
> > > > > >>                    <property name=3D"persistent" value=3D"true=
" />
> > > > > >>                </bean>
> > > > > >>            </list>
> > > > > >>        </property>
> > > > > >>    </bean>
> > > > > >> </property>
> > > > > >>
> > > > > >>
> > > > > >> Makes sense?
> > > > > >>
> > > > > >> Vladimir.
> > > > > >>
> > > > > >>
> > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> > > > > dsetrakyan@apache.org>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> Firstly all, why not call it DataPolicy instead of
> MemoryPolicy?
> > > > > >> Secondly,
> > > > > >>> why not set data policies directly on IgniteConfiguration. An=
d
> > > > lastly,
> > > > > >> how
> > > > > >>> about we combine memory and disk properties in one bean with
> > clear
> > > > > naming
> > > > > >>> convention?
> > > > > >>>
> > > > > >>> Here is the example. Note that all properties above must star=
t
> > with
> > > > > with
> > > > > >>> "Memory" or "Disk".
> > > > > >>>
> > > > > >>> *IgniteConfiguration cfg =3D new IgniteConfiguration();*
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > > > > >>>> .setName("bla"),        .setMemoryMaxSize(1024), // must be
> > > greater
> > > > > >> than
> > > > > >>> 0,
> > > > > >>>> since memory always needs to be enabled.
> > > .setDiskMaxSize(0),
> > > > //
> > > > > >> if
> > > > > >>>> greater than 0, then persistence is enabled.    );*
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>> I think this approach is much more concise and straight
> forward.
> > > What
> > > > > do
> > > > > >>> you think?
> > > > > >>>
> > > > > >>> D.
> > > > > >>>
> > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <
> > > > vozerov@gridgain.com
> > > > > >
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>> I prefer the second. Composition over inheritance - this is
> how
> > > all
> > > > > our
> > > > > >>>> configuration is crafted.
> > > > > >>>>
> > > > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > >>>> Instead, we have "CacheConfiguration.setCacheStoreFactory".
> > > > > >>>>
> > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > >>>>
> > > > > >>>>> Reiterating this based on some feedback from PDS users.
> > > > > >>>>>
> > > > > >>>>> It might be confusing to configure persistence with
> > > "MemoryPolicy",
> > > > > >> so
> > > > > >>>>> another approach is to deprecate the old names and introduc=
e
> a
> > > new
> > > > > >> name
> > > > > >>>>> "DataRegion" because it reflects the actual state when data
> is
> > > > stored
> > > > > >>> on
> > > > > >>>>> disk and partially in memory. I have two options in mind,
> each
> > of
> > > > > >> them
> > > > > >>>>> looks acceptable to me, so I would like to have some feedba=
ck
> > > from
> > > > > >> the
> > > > > >>>>> community. Old configuration names will be deprecated (but
> > still
> > > be
> > > > > >>> taken
> > > > > >>>>> if used for backward compatibility). Note, that old names
> > > > deprecation
> > > > > >>>>> handles default configuration compatibility very nicely -
> > current
> > > > PDS
> > > > > >>>> users
> > > > > >>>>> will not need to change anything to keep everything working=
.
> > The
> > > > two
> > > > > >>>>> options I mentioned are below:
> > > > > >>>>>
> > > > > >>>>> * we have two separate classes for in-memory and persisted
> data
> > > > > >>> regions,
> > > > > >>>>> so the configuration would look like so:
> > > > > >>>>>
> > > > > >>>>> IgniteConfiguration cfg =3D new IgniteConfiguration();
> > > > > >>>>>
> > > > > >>>>> cfg.setDataRegionsConfiguration(new
> DataRegionsConfiguration()
> > > > > >>>>>    .setDataRegions(
> > > > > >>>>>        new MemoryDataRegion()
> > > > > >>>>>            .setName("volatileCaches")
> > > > > >>>>>            .setMaxMemorySize(...),
> > > > > >>>>>        new PersistentDataRegion()
> > > > > >>>>>            .setName("persistentCaches")
> > > > > >>>>>            .setMaxMemorySize(...)
> > > > > >>>>>            .setMaxDiskSize()));
> > > > > >>>>>
> > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > >> PersistentStoreConfiguration()
> > > > > >>> );
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>> * we have one class for data region configuration, but it
> will
> > > > have a
> > > > > >>>>> sub-bean for persistence configuration:
> > > > > >>>>>
> > > > > >>>>> IgniteConfiguration cfg =3D new IgniteConfiguration();
> > > > > >>>>>
> > > > > >>>>> cfg.setDataRegionsConfiguration(new
> DataRegionsConfiguration()
> > > > > >>>>>    .setDataRegions(
> > > > > >>>>>        new DataRegion()
> > > > > >>>>>            .setName("volatileCaches")
> > > > > >>>>>            .setMaxMemorySize(...),
> > > > > >>>>>        new DataRegion()
> > > > > >>>>>            .setName("persistentCaches")
> > > > > >>>>>            .setMaxMemorySize(...),
> > > > > >>>>>            .setPersistenceConfiguration(
> > > > > >>>>>                new DataRegionPersistenceConfiguration()
> > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > >>>>>
> > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > >> PersistentStoreConfiguration()
> > > > > >>> );
> > > > > >>>>>
> > > > > >>>>
> > > > > >>>
> > > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

--94eb2c122f06072f55055a0883e6--