hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8401) Memfs - a layered file system for in-memory storage in HDFS
Date Thu, 14 May 2015 18:18:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14544138#comment-14544138
] 

Bikas Saha commented on HDFS-8401:
----------------------------------

I am guessing that anything that is written to in-memory storage via memfs would also be read
back in-memory via memfs. The centralized cache management is not the only in-memory read
path, right?

> Memfs - a layered file system for in-memory storage in HDFS
> -----------------------------------------------------------
>
>                 Key: HDFS-8401
>                 URL: https://issues.apache.org/jira/browse/HDFS-8401
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>
> We propose creating a layered filesystem that can provide in-memory storage using existing
features within HDFS. memfs will use lazy persist writes introduced by HDFS-6581. For reads,
memfs can use the Centralized Cache Management feature introduced in HDFS-4949 to load hot
data to memory.
> Paths in memfs and hdfs will correspond 1:1 so memfs will require no additional metadata
and it can be implemented entirely as a client-side library.
> The advantage of a layered file system is that it requires little or no changes to existing
applications. e.g. Applications can use something like {{memfs://}} instead of {{hdfs://}}
for files targeted to memory storage. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message