hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Bo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9040) Erasure coding: A BlockGroupDataStreamer to rule all internal blocks streamers
Date Mon, 14 Sep 2015 08:56:46 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14743182#comment-14743182
] 

Li Bo commented on HDFS-9040:
-----------------------------

Thanks Walter and Jing ‘s work.
In Jing’s patch, {{allocateNewBlock}} is called in {{writeChunk()}}, which makes the packets
in the {{dataQueue}} of streamer all belong to the same block. I think this is a good way
to reduce the logic complexity.  Currently the fastest streamer also has to wait for other
streamers before requesting a following block group from NN, so I think we may not feel the
writing speed becomes slow.


> Erasure coding: A BlockGroupDataStreamer to rule all internal blocks streamers
> ------------------------------------------------------------------------------
>
>                 Key: HDFS-9040
>                 URL: https://issues.apache.org/jira/browse/HDFS-9040
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Walter Su
>            Assignee: Walter Su
>         Attachments: HDFS-9040.00.patch, HDFS-9040.001.wip.patch
>
>
> A {{BlockGroupDataStreamer}} to communicate with NN to allocate/update block, and {{StripedDataStreamer}}
s only have to stream blocks to DNs. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message