Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
From: Denis Magda <dmagda@apache.org>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Data eviction/expiration from Ignite persistence
Message-Id: <2C96ED59-8CCF-4377-82FD-D2340EFF8C36@apache.org>
Date: Fri, 17 Nov 2017 13:40:59 -0800
To: dev@ignite.apache.org
archived-at: Fri, 17 Nov 2017 21:41:02 -0000

Igniters,

I=E2=80=99ve been talking to many Ignite users here and there who are =
already on Ignite persistence or consider to turn it on. The majority of =
them are more than satisfied with its current state and provided =
capabilities. That=E2=80=99s is really good news for us.

However, I tend to come across the people who ask about =
eviction/expiration policies for the persistence itself. Had around 6 =
conversation about the topic this month only.

Usually the requirement is connected with a streaming use case. When an =
application streams a lot of data (IoT, metrics, etc.) to the cluster =
but the data becomes stale in some period of time (day, couple of days, =
etc.). The user doesn=E2=80=99t want to waste the disk space and needs =
to simple purge the data from there.

My suggestion here is to create a timer task that will remove the stale =
data from the cluster. However, since the demand is growing probably =
it=E2=80=99s a good time to discuss a feasibility of this feature.

Alex G, as the main architect of the persistence, could you share your =
thoughts on this? What will it cost to us to support eviction/expiration =
for the persistence?

=E2=80=94
Denis=20=