hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-4953) enable HDFS local reads via mmap
Date Wed, 03 Jul 2013 01:11:20 GMT
Colin Patrick McCabe created HDFS-4953:

             Summary: enable HDFS local reads via mmap
                 Key: HDFS-4953
                 URL: https://issues.apache.org/jira/browse/HDFS-4953
             Project: Hadoop HDFS
          Issue Type: New Feature
    Affects Versions: 2.2.0
            Reporter: Colin Patrick McCabe
            Assignee: Colin Patrick McCabe

Currently, the short-circuit local read pathway allows HDFS clients to access files directly
without going through the DataNode.  However, all of these reads involve a copy at the operating
system level, since they rely on the read() / pread() / etc family of kernel interfaces.

We would like to enable HDFS to read local files via mmap.  This would enable truly zero-copy

In the initial implementation, zero-copy reads will only be performed when checksums were
disabled.  Later, we can use the DataNode's cache awareness to only perform zero-copy reads
when we know that checksum has already been verified.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message