hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1391) Exiting safemode takes a long time when there are lots of blocks in the HDFS
Date Sat, 19 Feb 2011 15:32:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12996791#comment-12996791
] 

Hadoop QA commented on HDFS-1391:
---------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12469119/excessReplicas3.txt
  against trunk revision 1072023.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 21 new or modified tests.

    -1 patch.  The patch command could not apply the patch.

Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/187//console

This message is automatically generated.

> Exiting safemode takes a long time when there are lots of blocks in the HDFS
> ----------------------------------------------------------------------------
>
>                 Key: HDFS-1391
>                 URL: https://issues.apache.org/jira/browse/HDFS-1391
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: excessReplicas.1_trunk.txt, excessReplicas2.txt, excessReplicas3.txt
>
>
> When the namenode decides to exit safemode,  it acquires the FSNamesystem lock and then
iterates over all blocks in the blocksmap to determine if any block has any excess replicas.
This call takes upwards of 5 minutes on a cluster that has 100 million blocks. This delays
namenode restart to a good extent.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message