hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9079) Erasure coding: preallocate multiple generation stamps when creating striped blocks
Date Mon, 14 Sep 2015 21:21:45 GMT
Zhe Zhang created HDFS-9079:
-------------------------------

             Summary: Erasure coding: preallocate multiple generation stamps when creating
striped blocks
                 Key: HDFS-9079
                 URL: https://issues.apache.org/jira/browse/HDFS-9079
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Zhe Zhang


A non-striped DataStreamer goes through the following steps in error handling:
{code}
1) Finds error => 2) Asks NN for new GS => 3) Gets new GS from NN => 4) Applies new
GS to DN (createBlockOutputStream) => 5) Ack from DN => 6) Updates block on NN
{code}
To simplify the above we can preallocate GS when NN creates a new striped block group ({{FSN#createNewBlock}}).
For each new striped block group we can reserve {{NUM_PARITY_BLOCKS}} GS's. Then steps 1~3
in the above sequence can be saved. If more than {{NUM_PARITY_BLOCKS}} errors have happened
we shouldn't try to further recover anyway.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message