ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry Pavlov <dpavlov....@gmail.com>
Subject Re: Data eviction/expiration from Ignite persistence
Date Fri, 09 Mar 2018 11:43:05 GMT
As far as I know there is no plans. Denis please correct me if I'm wrong.

But users found these or similar bugs, it seems we need to support PDS and
TTL.

пт, 9 мар. 2018 г., 4:36 Valentin Kulichenko <valentin.kulichenko@gmail.com
>:

> Guys,
>
> What is the result of this discussion? Do we still not support eviction and
> expiration on persistence level? If so, any plans to change this?
>
> -Val
>
> On Tue, Nov 21, 2017 at 9:45 AM, Denis Magda <dmagda@apache.org> wrote:
>
> > We might break the compatibility for the next major release or even
> create
> > a tool that will migrate persistence files from an old to new formats.
> >
> > —
> > Denis
> >
> > > On Nov 21, 2017, at 8:34 AM, Dmitry Pavlov <dpavlov.spb@gmail.com>
> > wrote:
> > >
> > > Hi Denis,
> > >
> > > Second fix we need to do is B+ tree separation in per-partition basis:
> > > https://issues.apache.org/jira/browse/IGNITE-5874
> > >
> > > Should we take into account compatibilty issues with previous Ignite
> > > persistent store versions, because current TTL tree is persisted, and
> > will
> > > change its format?
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > >
> > > вт, 21 нояб. 2017 г. в 2:13, Denis Magda <dmagda@apache.org>:
> > >
> > >> Dmitriy,
> > >>
> > >> That’s about TTL and eviction support for Ignite persistence.
> Presently
> > if
> > >> you set an expiration or eviction policy for a cache it will be
> applied
> > for
> > >> data stored in memory. The policy never affects the persistence layer.
> > >>
> > >> —
> > >> Denis
> > >>
> > >>> On Nov 20, 2017, at 9:29 AM, Dmitry Pavlov <dpavlov.spb@gmail.com>
> > >> wrote:
> > >>>
> > >>> Hi Denis,
> > >>>
> > >>> Is this need covered by PDS + TTL?
> > >>>
> > >>> For the very first TTL test, I found some delay after applying TTL
> with
> > >> the
> > >>> repository enabled:
> https://issues.apache.org/jira/browse/IGNITE-6964
> > >>>
> > >>> And I'm wondering if the user's needs are covered by
> > >>> https://apacheignite.readme.io/docs/expiry-policies plus
> > >>> https://apacheignite.readme.io/docs/distributed-persistent-store
> > >>>
> > >>> Sincerely,
> > >>> Dmitriy Pavlov
> > >>>
> > >>> сб, 18 нояб. 2017 г. в 12:12, Dmitry Pavlov <dpavlov.spb@gmail.com>:
> > >>>
> > >>>> Hi Denis,
> > >>>>
> > >>>> What is the difference of required by users functionality with
TTL
> > cache
> > >>>> expiration?
> > >>>>
> > >>>> By some posts I can suppose TTL cache is compatible with native
> > >>>> persistence.
> > >>>>
> > >>>> Sincerely,
> > >>>> Dmitriy Pavlov
> > >>>>
> > >>>> сб, 18 нояб. 2017 г. в 0:41, Denis Magda <dmagda@apache.org>:
> > >>>>
> > >>>>> Igniters,
> > >>>>>
> > >>>>> I’ve been talking to many Ignite users here and there who
are
> already
> > >> on
> > >>>>> Ignite persistence or consider to turn it on. The majority
of them
> > are
> > >> more
> > >>>>> than satisfied with its current state and provided capabilities.
> > >> That’s is
> > >>>>> really good news for us.
> > >>>>>
> > >>>>> However, I tend to come across the people who ask about
> > >>>>> eviction/expiration policies for the persistence itself. Had
> around 6
> > >>>>> conversation about the topic this month only.
> > >>>>>
> > >>>>> Usually the requirement is connected with a streaming use case.
> When
> > an
> > >>>>> application streams a lot of data (IoT, metrics, etc.) to the
> cluster
> > >> but
> > >>>>> the data becomes stale in some period of time (day, couple
of days,
> > >> etc.).
> > >>>>> The user doesn’t want to waste the disk space and needs to
simple
> > >> purge the
> > >>>>> data from there.
> > >>>>>
> > >>>>> My suggestion here is to create a timer task that will remove
the
> > stale
> > >>>>> data from the cluster. However, since the demand is growing
> probably
> > >> it’s a
> > >>>>> good time to discuss a feasibility of this feature.
> > >>>>>
> > >>>>> Alex G, as the main architect of the persistence, could you
share
> > your
> > >>>>> thoughts on this? What will it cost to us to support
> > >> eviction/expiration
> > >>>>> for the persistence?
> > >>>>>
> > >>>>> —
> > >>>>> Denis
> > >>>>
> > >>>>
> > >>
> > >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message