hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Feilong He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives
Date Mon, 11 Mar 2019 08:05:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789264#comment-16789264

Feilong He commented on HDFS-13762:

Thanks everyone for your valuable comments on this work. After discussed with [~umamaheswararao]
& [~rakeshr], we decide to split this JIRA into 4 subtasks as the above subtask section
shows. A big change to this work is that we will add an implementation of SCM cache by using
mapped byte buffer, which will not depend on PMDK. And the previous implementation with PMDK
is still kept. Using which implementation is configurable. Currently, the code is under refining.
I will upload the patches in these days.

> Support non-volatile storage class memory(SCM) in HDFS cache directives
> -----------------------------------------------------------------------
>                 Key: HDFS-13762
>                 URL: https://issues.apache.org/jira/browse/HDFS-13762
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: caching, datanode
>            Reporter: Sammi Chen
>            Assignee: Feilong He
>            Priority: Major
>         Attachments: HDFS-13762.000.patch, HDFS-13762.001.patch, HDFS-13762.002.patch,
HDFS-13762.003.patch, HDFS-13762.004.patch, HDFS-13762.005.patch, HDFS-13762.006.patch, HDFS-13762.007.patch,
HDFS-13762.008.patch, SCMCacheDesign-2018-11-08.pdf, SCMCacheTestPlan.pdf
> No-volatile storage class memory is a type of memory that can keep the data content after
power failure or between the power cycle. Non-volatile storage class memory device usually
has near access speed as memory DIMM while has lower cost than memory.  So today It is usually
used as a supplement to memory to hold long tern persistent data, such as data in cache. 
> Currently in HDFS, we have OS page cache backed read only cache and RAMDISK based lazy
write cache.  Non-volatile memory suits for both these functions. 
> This Jira aims to enable storage class memory first in read cache. Although storage class
memory has non-volatile characteristics, to keep the same behavior as current read only cache,
we don't use its persistent characteristics currently.  

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message