ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexey Goncharuk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-3477) Rework offheap storage
Date Thu, 14 Jul 2016 21:35:20 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378432#comment-15378432

Alexey Goncharuk commented on IGNITE-3477:

I think snapshot should succeed as long as we have all partitions backed up. If we loose a
partition, the snapshot should fail.

> Rework offheap storage
> ----------------------
>                 Key: IGNITE-3477
>                 URL: https://issues.apache.org/jira/browse/IGNITE-3477
>             Project: Ignite
>          Issue Type: Task
>          Components: cache
>            Reporter: Alexey Goncharuk
>            Assignee: Alexey Goncharuk
>             Fix For: 2.0
> Current (Ignite 1.x) implementation of cache storage relies on a tiered approach where
cache entry can be located in one of the three storage areas: on-heap, off-heap memory and
swap. It leads to the following disadvantages:
>  * Entry constantly moves from one storage area to another which leads to a complex code
(for example, swap/unswap listeners for queries and rebalancing)
>  * It is not possible to set per-cache memory limit
>  * Off-heap indexes require on-heap row cache. If this cache is small, performance becomes
very poor
>  * Continuous put/remove operations with OFFHEAP_TIERED mode lead to uncontrollable memory
> We can reapproach the cache storage and base it on a concept of page memory. We will
allocate several memory pools of pages of fixed size and assign each cache to one of the memory
pools. All storage data structures should operate on memory pages, the main storage will be
always off-heap with an optional on-heap cache.
> This gives us the following benefits:
>  * Flexible and precise per-cache memory limit
>  * Transparent swap: we can allocate page memory over a memory-mapped file and OS will
take care of swapping
>  * Getting rid of on-heap cache for off-heap SQL indexes
>  * Ability to take a cluster-wide data snapshot

This message was sent by Atlassian JIRA

View raw message