hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7980) Incremental BlockReport will dramatically slow down the startup of a namenode
Date Wed, 25 Mar 2015 08:22:52 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379484#comment-14379484
] 

Tsz Wo Nicholas Sze commented on HDFS-7980:
-------------------------------------------

As mentioned by [~huizane] in the description, the problem seems that blockReport() may send
an incremental block report by reportReceivedDeletedBlocks() before sending the full report.
 In that case, NN treats the incremental block report as the first block report but not the
full report.  The processing time for the full report is increased dramatically.

> Incremental BlockReport will dramatically slow down the startup of  a namenode
> ------------------------------------------------------------------------------
>
>                 Key: HDFS-7980
>                 URL: https://issues.apache.org/jira/browse/HDFS-7980
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Hui Zheng
>            Assignee: Walter Su
>
> In the current implementation the datanode will call the reportReceivedDeletedBlocks()
method that is a IncrementalBlockReport before calling the bpNamenode.blockReport() method.
So in a large(several thousands of datanodes) and busy cluster it will slow down(more than
one hour) the startup of namenode. 
> {code}
> List<DatanodeCommand> blockReport() throws IOException {
>     // send block report if timer has expired.
>     final long startTime = now();
>     if (startTime - lastBlockReport <= dnConf.blockReportInterval) {
>       return null;
>     }
>     final ArrayList<DatanodeCommand> cmds = new ArrayList<DatanodeCommand>();
>     // Flush any block information that precedes the block report. Otherwise
>     // we have a chance that we will miss the delHint information
>     // or we will report an RBW replica after the BlockReport already reports
>     // a FINALIZED one.
>     reportReceivedDeletedBlocks();
>     lastDeletedReport = startTime;
>     .........
>         // Send the reports to the NN.
>     int numReportsSent = 0;
>     int numRPCs = 0;
>     boolean success = false;
>     long brSendStartTime = now();
>     try {
>       if (totalBlockCount < dnConf.blockReportSplitThreshold) {
>         // Below split threshold, send all reports in a single message.
>         DatanodeCommand cmd = bpNamenode.blockReport(
>             bpRegistration, bpos.getBlockPoolId(), reports);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message