hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Prakash (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4075) Reduce recommissioning overhead
Date Mon, 22 Oct 2012 18:30:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481587#comment-13481587

Ravi Prakash commented on HDFS-4075:

Hi Kihwal!

The log message is

bq. LOG.info("Invalidated" + numOverReplicated + " over-replicated blocks on " +
bq.        srcNode + " during recommissioning");

which might mislead me to believe that the block invalidated was on srcNode, when it could
be any one of the 4 nodes. Maybe something to the effect "Recommissioning of srcNode led to
numOverReplicated over-replicated blocks to be invalidated"? 

Can you please also explain the change in DatanodeManager.java in this patch? node.isAlive
will be updated only when the node heartbeats in. So when will blockManager.processOverReplicatedBlocksOnReCommission(node);
be called?

> Reduce recommissioning overhead
> -------------------------------
>                 Key: HDFS-4075
>                 URL: https://issues.apache.org/jira/browse/HDFS-4075
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.23.4, 2.0.2-alpha
>            Reporter: Kihwal Lee
>            Assignee: Kihwal Lee
>            Priority: Critical
>         Attachments: hdfs-4075.patch
> When datanodes are recommissioned, {BlockManager#processOverReplicatedBlocksOnReCommission()}
is called for each rejoined node and excess blocks are added to the invalidate list. The problem
is this is done while the namesystem write lock is held.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message