ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Denis Magda (Jira)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-10862) Introduce tool to free up space on a disc of unused memory pages
Date Fri, 13 Sep 2019 16:40:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-10862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929354#comment-16929354

Denis Magda commented on IGNITE-10862:

[~agoncharuk] is this a fundamental problem in our architecture? If not to know all the details
it sounds like each node needs to spawn a background thread that will be cleaning disk space.

> Introduce tool to free up space on a disc of unused memory pages
> ----------------------------------------------------------------
>                 Key: IGNITE-10862
>                 URL: https://issues.apache.org/jira/browse/IGNITE-10862
>             Project: Ignite
>          Issue Type: Improvement
>          Components: persistence
>    Affects Versions: 2.7
>            Reporter: Vyacheslav Daradur
>            Priority: Critical
>             Fix For: 2.8
> At the current time, Ignite is not able to release occupied disc space even if a significant
amount of data was removed from IgniteCache. It disturbs our users because they can't use
hardware effectively, also it increases TCO.
> *Use-cases:*
> Let's imagine that we have used IgniteCache with enabled PDS during the time:
> - hardware disc space has been occupied during growing up of an amount
> of data, e.g. 100Gb;
> - then, we removed non-actual data, e.g 50Gb, which became useless for us;
> - disc space stopped growing up with new data, but it was not
> released, and still took 100Gb, instead of expected 50Gb;
> Another use case:
> - a user extracts data from IgniteCache to store it in separate
> IgniteCache or another store;
> - disc still is occupied and the user is not able to store data in the
> different cache at the same cluster because of disc limitation;
> *Possible solutions:*
> Introduce some kind of mechanics which will be able to shrink data in IgniteCache and
to free up space on a disc of mapped free memory pages.
> A solution should take into account possible concurrent reads and updates of data and
indexes to maintain consistency.
> Maybe, it makes sense to implement a tool to be offline at the first step that means
concurrent operations will be prohibited. 
> Strongly recommended discussing design on dev-list with Ignite experts before the start
of implementation.

This message was sent by Atlassian Jira

View raw message