hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaoyu Yao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7291) Persist in-memory replicas with appropriate unbuffered copy API on POSIX and Windows
Date Tue, 28 Oct 2014 16:37:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14187057#comment-14187057
] 

Xiaoyu Yao commented on HDFS-7291:
----------------------------------

The Jenkins failure is related to the change from HADOOP-10926 on test-patch.sh. 
An earlier break was found by HADOOP-11240 and resolved. I attached the new failures to HADOOP-10926.





-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.


> Persist in-memory replicas with appropriate unbuffered copy API on POSIX and Windows
> ------------------------------------------------------------------------------------
>
>                 Key: HDFS-7291
>                 URL: https://issues.apache.org/jira/browse/HDFS-7291
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>    Affects Versions: 2.6.0
>            Reporter: Xiaoyu Yao
>            Assignee: Xiaoyu Yao
>         Attachments: HDFS-7291.0.patch, HDFS-7291.1.patch, HDFS-7291.2.patch, HDFS-7291.3.patch,
HDFS-7291.4.patch
>
>
> HDFS-7090 changes to persist in-memory replicas using unbuffered IO on Linux and Windows.
On Linux distribution, it relies on the sendfile() API between two file descriptors to achieve
unbuffered IO copy. According to Linux document at http://man7.org/linux/man-pages/man2/sendfile.2.html,
this is only supported on Linux kernel 2.6.33+. 
> As pointed by Haowei in the discussion below, FileChannel#transferTo already has support
for native unbuffered IO on POSIX platform. On Windows, JDK 6/7/8 has not implemented native
unbuffered IO yet. We change to use FileChannel#transfer for POSIX and our own native wrapper
of CopyFileEx on Windows for unbuffered copy.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message