hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-7854) Separate class DataStreamer out of DFSOutputStream
Date Thu, 12 Mar 2015 05:14:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358106#comment-14358106
] 

Jing Zhao edited comment on HDFS-7854 at 3/12/15 5:14 AM:
----------------------------------------------------------

Thanks for the response, Bo. For 5&6, after separating DataStreamer out, DFSOutputStream
should not know/use the dataQueue anymore, since this is the internal details hidden inside
of DataStreamer.

Thus DataStreamer should maintain its own queue and provide corresponding {{queueCurrentPacket}}
and {{waitAndQueueCurrentPacket}} methods for DFSOutputStream. My earlier [patch|https://issues.apache.org/jira/secure/attachment/12698643/HDFS-7793.000.patch]
also demos this idea. In this way it will be more clean and easier to maintain multiple DataStreamers
for a striped DFSOutputStream. 


was (Author: jingzhao):
Thanks for the response, Bo. For 5&6, after separating DataStreamer out, DFSOutputStream
should not know/use the dataQueue anymore, since this is the internal details hidden inside
of DataStreamer. Instead, DataStreamer should define {{queueCurrentPacket}} and {{waitAndQueueCurrentPacket}}
methods for DFSOutputStream. My earlier [patch|https://issues.apache.org/jira/secure/attachment/12698643/HDFS-7793.000.patch]
also demos this idea.

> Separate class DataStreamer out of DFSOutputStream
> --------------------------------------------------
>
>                 Key: HDFS-7854
>                 URL: https://issues.apache.org/jira/browse/HDFS-7854
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Li Bo
>            Assignee: Li Bo
>         Attachments: HDFS-7854-001.patch, HDFS-7854-002.patch, HDFS-7854-003.patch
>
>
> This sub task separate DataStreamer from DFSOutputStream. New DataStreamer will accept
packets and write them to remote datanodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message