hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Bo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7854) Separate class DataStreamer out of DFSOutputStream
Date Thu, 12 Mar 2015 04:27:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358081#comment-14358081

Li Bo commented on HDFS-7854:

Thanks for Jing's careful review. I will upload a new patch later.

1.	Yes, we can also keep these fields in DFSOutputStream. But I think it’d better to put
them in DataStreamer. DataStreamer is responsible for writing blocks, while DFSOutputStream
just puts packets into DataStreamer. And if there’re multiple DataStreamers in DFSOutputStream,
it’s not necessary to maintain an array of each field in DFSOutputStream .
2.	Will fix it in new patch.
3.	Will fix it in new patch.
4.	Will fix it in new patch. 
5 & 6 Because DFSOutputStream also needs to add packets to dataQueue, I think it’d better
to let DFSOutputStream maintain these two queues and transfer them to DataStreamer via its
7. We have moved several fields from DFSOutputStream to DataStreamer, after {{streamer = null}},
these fields can’t be accessed. So a new field {{toTerminate}} is imported. {{toTerminate
= true}} equals to {{streamer = null}}, {{if(\!toTerminate)}} equals to {{if(streamer!=null)}}.
If we use {{streamerClosed}} instead of {{toTerminate}},  I think we may change the original
logic and cause new errors(I will run unit tests later to see if the replacement will cause
new errors and why). 
8. {{DFSOupuStream.src}}: this is a way to avoid the findbugs warning, will fix it in new

> Separate class DataStreamer out of DFSOutputStream
> --------------------------------------------------
>                 Key: HDFS-7854
>                 URL: https://issues.apache.org/jira/browse/HDFS-7854
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Li Bo
>            Assignee: Li Bo
>         Attachments: HDFS-7854-001.patch, HDFS-7854-002.patch, HDFS-7854-003.patch
> This sub task separate DataStreamer from DFSOutputStream. New DataStreamer will accept
packets and write them to remote datanodes.

This message was sent by Atlassian JIRA

View raw message