ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mulugeta Mammo (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-6854) Enabling Persistent Memory for Ignite
Date Thu, 09 Nov 2017 23:34:00 GMT
Mulugeta Mammo created IGNITE-6854:

             Summary: Enabling Persistent Memory for Ignite
                 Key: IGNITE-6854
                 URL: https://issues.apache.org/jira/browse/IGNITE-6854
             Project: Ignite
          Issue Type: New Feature
      Security Level: Public (Viewable by anyone)
    Affects Versions: 2.3
            Reporter: Mulugeta Mammo
             Fix For: 2.4

Ignite, when persistence mode is enabled, stores data and indexes on disk. To minimize the
latency of disks, several tuning options can be applied. Setting the page size of a memory
region to match the page size of the underlying storage, using a separate disk for the WAL,
and using production-level SSDs are just a few of them [ https://apacheignite.readme.io/docs/durable-memory-tuning#section-native-persistence-related-tuning
A persistent memory store with low latency and high capacity offers a viable alternative to
disks. In light of this, we are proposing to make use of our Low Level Persistent Library
(LLPL), https://github.com/pmem/pcj/tree/master/LLPL, to offer a persistent memory storage
for Ignite. 
At this point, we envision two distinct implementation options:
# Data and indexes will continue to be stored in the off-heap memory but the disk will be
replaced by a persistent memory. Since persistence memory in this option is not a file system,
the logic currently offered by WAL file and the partition files would have to be implemented
from scratch.
# In this option, we eliminate the current check-point process and the WAL file. We will use
a memory region defined by LLPL to store data and indexes. There will be no off-heap memory.
DRAM will be exclusively used to store hot cache entries just like the on-heap cache is in
the current implementation. 
In both cases, there are more details and subtleties that have to handled – e.g. the atomic
and transactional guarantees offered. More clarifications will be offered as we go along.

This message was sent by Atlassian JIRA

View raw message