Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <2139926777.1210366675841.JavaMail.jira@brutus>
Date: Fri, 9 May 2008 13:57:55 -0700 (PDT)
From: "Hairong Kuang (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-3369) Fast block processing during
 name-node startup.
In-Reply-To: <738573588.1210291015541.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12595717#action_12595717 ] 

Hairong Kuang commented on HADOOP-3369:
---------------------------------------

Nice change! Simple but great startup performance improvement.

In FSnmensystem.processMisReplicatedBlocks, it is better to reset all kinds of queues in the very beginning, then simply add blocks to each queue with no need of removal.

> Fast block processing during name-node startup.
> -----------------------------------------------
>
>                 Key: HADOOP-3369
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3369
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.17.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.18.0
>
>         Attachments: fastBlockReports.patch
>
>
> The block report processing during the startup period should be optimized.
> As noted in HADOOP-3022 during cluster startup all blocks are under-replicated 
> because they have not been reported by name-nodes yet.
> Currently, we routinely move blocks to the neededReplications queue when they
> are first reported and then remove them from the list when other nodes report it.
> In ideal situation we end up adding all blocks into neededReplications queue first
> only in order to remove all of them in the end. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.