hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3051) A zero-copy ScatterGatherRead api from FSDataInputStream
Date Tue, 06 Mar 2012 09:10:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223116#comment-13223116
] 

Todd Lipcon commented on HDFS-3051:
-----------------------------------

One interesting thing to take note of: in Linux prior to 2.6.37, the page fault handler for
file mappings actually held the mmap semaphore exclusively, preventing other threads from
modifying page mappings (or starting threads). So doing mmapped IO may have some downsides
as well, especially on older kernels. Not sure if this issue is addressed in RHEL 6 or not.
The Linux git hash is d065bd810b6deb67d4897a14bfe21f8eb526ba99, see also http://help.lockergnome.com/linux/PATCH-V2-Reduce-mmap_sem-hold-times-file-backed-page-faults--ftopict527005.html
                
> A zero-copy ScatterGatherRead api from FSDataInputStream
> --------------------------------------------------------
>
>                 Key: HDFS-3051
>                 URL: https://issues.apache.org/jira/browse/HDFS-3051
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs client
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>
> It will be nice if we can get a new API from FSDtaInputStream that allows for zero-copy
read for hdfs readers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message