hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
Date Thu, 20 Feb 2014 11:14:56 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906871#comment-13906871
] 

Hudson commented on HDFS-5868:
------------------------------

SUCCESS: Integrated in Hadoop-Yarn-trunk #487 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/487/])
HDFS-5868. Make hsync implementation pluggable. (Contributed by Buddy Taylor) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1569978)
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/ReplicaOutputStreams.java


> Make hsync implementation pluggable
> -----------------------------------
>
>                 Key: HDFS-5868
>                 URL: https://issues.apache.org/jira/browse/HDFS-5868
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>    Affects Versions: 2.2.0
>            Reporter: Buddy
>             Fix For: 3.0.0, 2.4.0
>
>         Attachments: HDFS-5868-branch-2.patch, HDFS-5868a-branch-2.patch, HDFS-5868b-branch-2.patch
>
>
> The current implementation of hsync in BlockReceiver only works if the output streams
are instances of FileOutputStream. Therefore, there is currently no way for a FSDatasetSpi
plugin to implement hsync if it is not using standard OS files.
> One possible solution is to push the implementation of hsync into the ReplicaOutputStreams
class. This class is constructed by the ReplicaInPipeline which is constructed by the FSDatasetSpi
plugin, therefore it can be extended. Instead of directly calling sync on the output stream,
BlockReceiver would call ReplicaOutputStream.sync.  The default implementation of sync in
ReplicaOutputStream would be the same as the current implementation in BlockReceiver. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message