hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8401) Memfs - a layered file system for in-memory storage in HDFS
Date Thu, 14 May 2015 15:59:01 GMT
Arpit Agarwal created HDFS-8401:

             Summary: Memfs - a layered file system for in-memory storage in HDFS
                 Key: HDFS-8401
                 URL: https://issues.apache.org/jira/browse/HDFS-8401
             Project: Hadoop HDFS
          Issue Type: Bug
            Reporter: Arpit Agarwal
            Assignee: Arpit Agarwal

We propose creating a layered filesystem that can provide in-memory storage using existing
features within HDFS. memfs will use lazy persist writes introduced by HDFS-6581. For reads,
memfs can use the Centralized Cache Management feature introduced in HDFS-4949 to load hot
data to memory.

Paths in memfs and hdfs will correspond 1:1 so memfs will require no additional metadata and
it can be implemented entirely as a client-side library.

The advantage of a layered file system is that it requires little or no changes to existing
applications. e.g. Applications can use something like {{memfs://}} instead of {{hdfs://}}
for files targeted to memory storage. The layered file system idea was first proposed in HDFS-5851
by [~sanjay.radia].

More details to follow.

This message was sent by Atlassian JIRA

View raw message