Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
From: Denis Magda <dmagda@apache.org>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: Data eviction/expiration from Ignite persistence
Date: Mon, 20 Nov 2017 15:13:28 -0800
References: <2C96ED59-8CCF-4377-82FD-D2340EFF8C36@apache.org>
 <CALUCNEthN91ZJ+jwpAviU_V3wAUAHAaKnqLcEY=wJZ_o7QROyg@mail.gmail.com>
 <CALUCNEsfWH1x=Vrk-6zXfs9HDQWkGmHKB5+bH0rtaosaDr54rw@mail.gmail.com>
To: dev@ignite.apache.org
In-Reply-To: <CALUCNEsfWH1x=Vrk-6zXfs9HDQWkGmHKB5+bH0rtaosaDr54rw@mail.gmail.com>
Message-Id: <74B12DD3-6C79-418B-ACFB-4DEAA5D75B8C@apache.org>
archived-at: Mon, 20 Nov 2017 23:13:32 -0000

Dmitriy,

That=E2=80=99s about TTL and eviction support for Ignite persistence. =
Presently if you set an expiration or eviction policy for a cache it =
will be applied for data stored in memory. The policy never affects the =
persistence layer.

=E2=80=94
Denis

> On Nov 20, 2017, at 9:29 AM, Dmitry Pavlov <dpavlov.spb@gmail.com> =
wrote:
>=20
> Hi Denis,
>=20
> Is this need covered by PDS + TTL?
>=20
> For the very first TTL test, I found some delay after applying TTL =
with the
> repository enabled: https://issues.apache.org/jira/browse/IGNITE-6964
>=20
> And I'm wondering if the user's needs are covered by
> https://apacheignite.readme.io/docs/expiry-policies plus
> https://apacheignite.readme.io/docs/distributed-persistent-store
>=20
> Sincerely,
> Dmitriy Pavlov
>=20
> =D1=81=D0=B1, 18 =D0=BD=D0=BE=D1=8F=D0=B1. 2017 =D0=B3. =D0=B2 12:12, =
Dmitry Pavlov <dpavlov.spb@gmail.com>:
>=20
>> Hi Denis,
>>=20
>> What is the difference of required by users functionality with TTL =
cache
>> expiration?
>>=20
>> By some posts I can suppose TTL cache is compatible with native
>> persistence.
>>=20
>> Sincerely,
>> Dmitriy Pavlov
>>=20
>> =D1=81=D0=B1, 18 =D0=BD=D0=BE=D1=8F=D0=B1. 2017 =D0=B3. =D0=B2 0:41, =
Denis Magda <dmagda@apache.org>:
>>=20
>>> Igniters,
>>>=20
>>> I=E2=80=99ve been talking to many Ignite users here and there who =
are already on
>>> Ignite persistence or consider to turn it on. The majority of them =
are more
>>> than satisfied with its current state and provided capabilities. =
That=E2=80=99s is
>>> really good news for us.
>>>=20
>>> However, I tend to come across the people who ask about
>>> eviction/expiration policies for the persistence itself. Had around =
6
>>> conversation about the topic this month only.
>>>=20
>>> Usually the requirement is connected with a streaming use case. When =
an
>>> application streams a lot of data (IoT, metrics, etc.) to the =
cluster but
>>> the data becomes stale in some period of time (day, couple of days, =
etc.).
>>> The user doesn=E2=80=99t want to waste the disk space and needs to =
simple purge the
>>> data from there.
>>>=20
>>> My suggestion here is to create a timer task that will remove the =
stale
>>> data from the cluster. However, since the demand is growing probably =
it=E2=80=99s a
>>> good time to discuss a feasibility of this feature.
>>>=20
>>> Alex G, as the main architect of the persistence, could you share =
your
>>> thoughts on this? What will it cost to us to support =
eviction/expiration
>>> for the persistence?
>>>=20
>>> =E2=80=94
>>> Denis
>>=20
>>=20