hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Feilong He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13762) Support non-volatile storage class memory(SCM) in HDFS cache directives
Date Tue, 26 Mar 2019 16:11:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16801880#comment-16801880

Feilong He commented on HDFS-13762:

I just uploaded SCMCacheDesign-2019-03-26.pdf. This design doc was updated to align with our
current implementation. We also added some future plans on HDFS SCM cache. Please refer to
this doc for more easily understanding our implementation.

In addition, currently we are working on this jira's subtask HDFS-14355. You can go there
to review our patch. Suggestions are always welcome. Thanks!

> Support non-volatile storage class memory(SCM) in HDFS cache directives
> -----------------------------------------------------------------------
>                 Key: HDFS-13762
>                 URL: https://issues.apache.org/jira/browse/HDFS-13762
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: caching, datanode
>            Reporter: Sammi Chen
>            Assignee: Feilong He
>            Priority: Major
>         Attachments: HDFS-13762.000.patch, HDFS-13762.001.patch, HDFS-13762.002.patch,
HDFS-13762.003.patch, HDFS-13762.004.patch, HDFS-13762.005.patch, HDFS-13762.006.patch, HDFS-13762.007.patch,
HDFS-13762.008.patch, SCMCacheDesign-2018-11-08.pdf, SCMCacheDesign-2019-3-26.pdf, SCMCacheTestPlan.pdf
> No-volatile storage class memory is a type of memory that can keep the data content after
power failure or between the power cycle. Non-volatile storage class memory device usually
has near access speed as memory DIMM while has lower cost than memory.  So today It is usually
used as a supplement to memory to hold long tern persistent data, such as data in cache. 
> Currently in HDFS, we have OS page cache backed read only cache and RAMDISK based lazy
write cache.  Non-volatile memory suits for both these functions. 
> This Jira aims to enable storage class memory first in read cache. Although storage class
memory has non-volatile characteristics, to keep the same behavior as current read only cache,
we don't use its persistent characteristics currently.  

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message