Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 27E53200D29 for ; Thu, 12 Oct 2017 07:06:07 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 26578160BE3; Thu, 12 Oct 2017 05:06:07 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1F4B51609E5 for ; Thu, 12 Oct 2017 07:06:05 +0200 (CEST) Received: (qmail 77203 invoked by uid 500); 12 Oct 2017 05:06:05 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 77191 invoked by uid 99); 12 Oct 2017 05:06:05 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Oct 2017 05:06:05 +0000 Received: from mail-qt0-f179.google.com (mail-qt0-f179.google.com [209.85.216.179]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id DDFD71A0117 for ; Thu, 12 Oct 2017 05:06:03 +0000 (UTC) Received: by mail-qt0-f179.google.com with SMTP id f15so11569675qtf.7 for ; Wed, 11 Oct 2017 22:06:03 -0700 (PDT) X-Gm-Message-State: AMCzsaXjJJ8JhzhNzK1fyvOdAV15+HWf/2BB8TthAcsJbo7+7FuAbHc1 hw2qIP1aQISNYo3uVEkSDei88hoZY8rfZnsw4YK5aw== X-Google-Smtp-Source: AOwi7QDx5QWkXMvSETS7gFNE9VSwttbgRQ+Q7LYzJ1ip3JsQE2HtBs7E2rxX/9w0M9/GkasBTZuKIyAYb2yiPQAARXU= X-Received: by 10.200.33.186 with SMTP id 55mr1897075qty.300.1507784761665; Wed, 11 Oct 2017 22:06:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.28.3 with HTTP; Wed, 11 Oct 2017 22:05:20 -0700 (PDT) In-Reply-To: References: <2e3f37db-4cd3-f595-2506-3c3db408fc61@gmail.com> <3C43D7EC-F109-480D-A71B-197F5A407DA0@apache.org> <5a1980fa-184c-d77d-9d1d-9a02ae6ce281@gmail.com> <70089e9a-caa9-9236-24ed-1daa6e1f9810@gmail.com> <605757bd-751f-5954-be26-be044c4244ce@gmail.com> <14cd3d61-daba-d2c1-8ef9-646c03a64b35@gmail.com> From: Dmitriy Setrakyan Date: Wed, 11 Oct 2017 22:05:20 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Persistence per memory policy configuration To: dev@ignite.apache.org Content-Type: multipart/alternative; boundary="001a113c3eb41df701055b527fa6" archived-at: Thu, 12 Oct 2017 05:06:07 -0000 --001a113c3eb41df701055b527fa6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Is the storage path the root folder for the persistence or only the root path for the main storage? On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda wrote: > Ivan, > > Instead of =E2=80=9CsetStoragePath=E2=80=9D I would suggest =E2=80=9CsetP= ersistencePath=E2=80=9D. Left > some extra notes in the ticket. > > =E2=80=94 > Denis > > > On Oct 11, 2017, at 4:30 AM, Ivan Rakov wrote: > > > > Vladimir, > > > > Thanks for focusing on existing namings. Most of your suggestions reall= y > sound better. I've posted my thoughts under your comment. > > > > By the way, we should decide two things: > > > > 1) Naming for methods for configuring store path. I suggest the > following: > > > > *setStoragePath* - for partition and index files > > *setWalPath* - for WAL files > > *walArchivePath* - for WAL archive files > > > > 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same wit= h > *checkpointingPageBufferSize* and *checkpointingThreads*). Both options > sounds ok to me, let's see what community thinks. > > > > Best Regards, > > Ivan Rakov > > > > On 11.10.2017 14:05, Vladimir Ozerov wrote: > >> Ivan, > >> > >> I left some comments in the ticket [1], please take a look. > >> > >> [1] > >> https://issues.apache.org/jira/browse/IGNITE-6030? > focusedCommentId=3D16200050&page=3Dcom.atlassian.jira. > plugin.system.issuetabpanels:comment-tabpanel#comment-16200050 > >> > >> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov > wrote: > >> > >>> Igniters, > >>> > >>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and > enqueued > >>> for TC run. > >>> PR: https://github.com/apache/ignite/pull/2828 > >>> > >>> Everyone interested in new data storage configuration API, please pay > >>> attention and review. > >>> > >>> > >>> Best Regards, > >>> Ivan Rakov > >>> > >>> > >>> On 09.10.2017 12:40, Pavel Tupitsyn wrote: > >>> > >>>> Sounds good to me. > >>>> > >>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov > >>>> wrote: > >>>> > >>>> Pavel, > >>>>> Sounds reasonable. > >>>>> I suggest to include both "data" and "configuration" to make it eve= n > more > >>>>> obvious: > >>>>> > >>>>> set/getDefaultDataRegionConfiguration > >>>>> set/getDataRegionConfigurations > >>>>> > >>>>> Best Regards, > >>>>> Ivan Rakov > >>>>> > >>>>> > >>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote: > >>>>> > >>>>> Sorry that I'm late to the party, but this looks inconsistent: > >>>>>> DataStorageConfiguration defaultRegionConfiguration > >>>>>> DataRegionConfiguration[] getDataRegions > >>>>>> > >>>>>> defaultRegionConfiguration + getRegionConfigurations > >>>>>> - or - > >>>>>> defaultDataRegion + getDataRegions > >>>>>> > >>>>>> Thoughts? > >>>>>> > >>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov > >>>>>> wrote: > >>>>>> > >>>>>> Denis, > >>>>>> > >>>>>>> Yes, you're right. All cache groups without specific data region > >>>>>>> configured will be persistent. > >>>>>>> And if you want to add another persistent data region, you should > set > >>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* > explictly. > >>>>>>> > >>>>>>> Best Regards, > >>>>>>> Ivan Rakov > >>>>>>> > >>>>>>> > >>>>>>> On 02.10.2017 21:01, Denis Magda wrote: > >>>>>>> > >>>>>>> Missed the point with defaults. Makes sense to me now. So to wrap > this > >>>>>>> > >>>>>>>> up, if I want to enable the persistence globally and don=E2=80= =99t have > any > >>>>>>>> regions > >>>>>>>> configured explicitly I need to take the default region and > switch the > >>>>>>>> persistence on for it. Is my understanding correct? > >>>>>>>> > >>>>>>>> =E2=80=94 > >>>>>>>> Denis > >>>>>>>> > >>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>> Denis, why do you need to access an instance of the default regi= on > >>>>>>>>> bean? > >>>>>>>>> If you want to set any parameter, just instantiate new bean wit= h > this > >>>>>>>>> parameter set (like in XML snipped below). Other parameters wil= l > be > >>>>>>>>> automatically initialized with their default values. > >>>>>>>>> > >>>>>>>>> Best Regards, > >>>>>>>>> Ivan Rakov > >>>>>>>>> > >>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> >>>>>>>>>>>> guration.DataStorageConfiguration"> > >>>>>>>>>>>> >>>>>>>>>>>> value=3D"#{100 > >>>>>>>>>>>> * > >>>>>>>>>>>> 1024 * 1024}"/> > >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>> guration.DataRegionConfiguration"> > >>>>>>>>>>>> >>>>>>>>>>>> 1024 * > >>>>>>>>>>>> 102 * 1024}"/> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> In other data regions persistence will be disabled by defaul= t. > >>>>>>>>>>>> > >>>>>>>>>>> Ivan, how to get an instance to the default region bean and > change > >>>>>>>>>>> a > >>>>>>>>>>> > >>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence= I > >>>>>>>>>> don=E2=80=99t want > >>>>>>>>>> to create the default region bean from scratch. > >>>>>>>>>> > >>>>>>>>>> =E2=80=94 > >>>>>>>>>> Denis > >>>>>>>>>> > >>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov > >>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> Agree with Alexey. > >>>>>>>>>>> Properties like *defaultDataRegionSize*, > >>>>>>>>>>> *isDefaultPersistenceEnabled* > >>>>>>>>>>> can confuse users who don't know that there's such thing as > default > >>>>>>>>>>> data > >>>>>>>>>>> region. They can decide they are inherited by all data region= s > >>>>>>>>>>> where > >>>>>>>>>>> size > >>>>>>>>>>> and persistence flag are not explicitly set. > >>>>>>>>>>> > >>>>>>>>>>> Let's get rid of these properties and add > >>>>>>>>>>> *defaultRegionConfiguration* > >>>>>>>>>>> property with explicit configuration of default data region. > >>>>>>>>>>> > >>>>>>>>>>> Regarding XML configuration, changing size or persistence fla= g > of > >>>>>>>>>>> default data region will be just two lines longer (for bean > >>>>>>>>>>> description): > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>>> guration.DataStorageConfiguration"> > >>>>>>>>>>>> >>>>>>>>>>>> value=3D"#{100 > >>>>>>>>>>>> * > >>>>>>>>>>>> 1024 * 1024}"/> > >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>> guration.DataRegionConfiguration"> > >>>>>>>>>>>> >>>>>>>>>>>> 1024 * > >>>>>>>>>>>> 102 * 1024}"/> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> In other data regions persistence will be disabled by defaul= t. > >>>>>>>>>>>> > >>>>>>>>>>> I've updated draft in https://issues.apache.org/jira > >>>>>>>>>>> /browse/IGNITE-6030 with these changes. > >>>>>>>>>>> > >>>>>>>>>>> Best Regards, > >>>>>>>>>>> Ivan Rakov > >>>>>>>>>>> > >>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote: > >>>>>>>>>>> > >>>>>>>>>>> To resolve this, I suggest to > >>>>>>>>>>> > >>>>>>>>>>>> introduce just another field defaultRegionConfiguration and > get > >>>>>>>>>>>>> rid > >>>>>>>>>>>>> of > >>>>>>>>>>>>> other defaults in DataStorageConfiguration. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Won=E2=80=99t it complicate the configuration from a Spring= XML > file? I=E2=80=99m > >>>>>>>>>>>>> > >>>>>>>>>>>> not > >>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfigurati= on > >>>>>>>>>>>> bean > >>>>>>>>>>>> first > >>>>>>>>>>>> to change any parameter? > >>>>>>>>>>>> > >>>>>>>>>>>> =E2=80=94 > >>>>>>>>>>>> Denis > >>>>>>>>>>>> > >>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk < > >>>>>>>>>>>> > >>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote: > >>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would > either > >>>>>>>>>>>>> need to > >>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on > >>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this fiel= d > >>>>>>>>>>>>> which > >>>>>>>>>>>>> is also > >>>>>>>>>>>>> an overkill. On the other hand, one can assume that the > defaults > >>>>>>>>>>>>> we > >>>>>>>>>>>>> are > >>>>>>>>>>>>> talking about are actually inherited. To resolve this, I > suggest > >>>>>>>>>>>>> to > >>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and > get > >>>>>>>>>>>>> rid > >>>>>>>>>>>>> of > >>>>>>>>>>>>> other defaults in DataStorageConfiguration. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thoughts? > >>>>>>>>>>>>> > >>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov >: > >>>>>>>>>>>>> > >>>>>>>>>>>>> Vladimir, > >>>>>>>>>>>>> > >>>>>>>>>>>>> I like your approach because it's easier to implement. > >>>>>>>>>>>>>> However, user may be confused by setting > >>>>>>>>>>>>>> *isDefaultPersistenceEnabled* > >>>>>>>>>>>>>> flag and seeing that persistence is not enabled by default > in > >>>>>>>>>>>>>> custom memory > >>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>> Ivan Rakov > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Ivan, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I do not think this is correct approach, because it will b= e > hard > >>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of > >>>>>>>>>>>>>>> "boolean" > >>>>>>>>>>>>>>> for > >>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default > >>>>>>>>>>>>>>> "persistence > >>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have > "persistence > >>>>>>>>>>>>>>> enabled" > >>>>>>>>>>>>>>> flag for default region only. It should not be propagated > to > >>>>>>>>>>>>>>> custom > >>>>>>>>>>>>>>> regions. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov < > >>>>>>>>>>>>>>> ivan.glukos@gmail.com > >>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Guys, I think I got the point now. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Let's check the final design: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> *DataStorageConfiguration* will have > >>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled* > >>>>>>>>>>>>>>>> property (default =3D false), which will be used for > enabling > >>>>>>>>>>>>>>>> persistence > >>>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>> default data region. > >>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnable= d* > >>>>>>>>>>>>>>>> property, > >>>>>>>>>>>>>>>> which > >>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding > data > >>>>>>>>>>>>>>>> region. If > >>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::is= D > >>>>>>>>>>>>>>>> efaultPersistenceEnabled* > >>>>>>>>>>>>>>>> will be used by default. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Best Regards, > >>>>>>>>>>>>>>>> Ivan Rakov > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda < > >>>>>>>>>>>>>>>> dmagda@apache.org> > >>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov < > ivan.glukos@gmail.com > >>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. > Persistence > >>>>>>>>>>>>>>>>>>> will be > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has th= is > >>>>>>>>>>>>>>>>>>> flag > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> set. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> I=E2=80=99m confused. Why the persistence should be en= abled > >>>>>>>>>>>>>>>>>> *globally* > >>>>>>>>>>>>>>>>>> if the > >>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it= =E2=80=99s > >>>>>>>>>>>>>>>>>> enabled for > >>>>>>>>>>>>>>>>>> region > >>>>>>>>>>>>>>>>>> A only, I don=E2=80=99t want to have it activated for = region B. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be > >>>>>>>>>>>>>>>>>> disabled > >>>>>>>>>>>>>>>>>> globally. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> But we should also give users a way to switch the > default > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> behavior from > >>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > > > > --001a113c3eb41df701055b527fa6--