ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Павлухин Иван <vololo...@gmail.com>
Subject Re: How to free up space on disc after removing entries from IgniteCache with enabled PDS?
Date Sat, 29 Dec 2018 07:46:59 GMT
I suppose it is about Ignite Page Memory pages defragmentation.

We can get 100 allocated pages each of which becomes only e.g. 50%
filled after removal some entries. But they will occupy a space for
100 pages on a hard drive.

пт, 28 дек. 2018 г. в 20:45, Denis Magda <dmagda@apache.org>:
>
> Shouldn't the OS care of defragmentation? What we need to do is to give a
> way to remove stale data and "release" the allocated space somehow through
> the tools, MBeans or API methods.
>
> --
> Denis
>
>
> On Fri, Dec 28, 2018 at 6:24 AM Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
>
> > Hi Vyacheslav,
> >
> > AFAIK this is not implemented. Shrinking/defragmentation is important
> > optimization. Not only because it releases free space, but also because it
> > decreases total number of pages. But is it not very easy to implement, as
> > you have to both reshuffle data entries and index entries, maintaining
> > consistency for concurrent reads and updates at the same time. Or
> > alternatively we can think of offline defragmentation. It will be easier to
> > implement and faster, but concurrent operations will be prohibited.
> >
> > On Fri, Dec 28, 2018 at 4:08 PM Vyacheslav Daradur <daradurvs@gmail.com>
> > wrote:
> >
> > > Igniters, we have faced with the following problem on one of our
> > > deployments.
> > >
> > > Let's imagine that we have used IgniteCache with enabled PDS during the
> > > time:
> > > - hardware disc space has been occupied during growing up of an amount
> > > of data, e.g. 100Gb;
> > > - then, we removed non-actual data, e.g 50Gb, which became useless for
> > us;
> > > - disc space stopped growing up with new data, but it was not
> > > released, and still took 100Gb, instead of expected 50Gb;
> > >
> > > Another use case:
> > > - a user extracts data from IgniteCache to store it in separate
> > > IgniteCache or another store;
> > > - disc still is occupied and the user is not able to store data in the
> > > different cache at the same cluster because of disc limitation;
> > >
> > > How can we help the user to free up the disc space, if an amount of
> > > data in IgniteCache has been reduced many times and will not be
> > > increased in the nearest future?
> > >
> > > AFAIK, we have mechanics of reusing memory pages, that allows us to
> > > use pages which have been allocated and stored removed data for
> > > storing new data.
> > > Are there any chances to shrink data and free up space on disc (with
> > > defragmentation if possible)?
> > >
> > > --
> > > Best Regards, Vyacheslav D.
> > >
> >



-- 
Best regards,
Ivan Pavlukhin

Mime
View raw message