hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1391) Exiting safemode takes a long time when there are lots of blocks in the HDFS
Date Wed, 24 Nov 2010 21:38:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935508#action_12935508
] 

Hadoop QA commented on HDFS-1391:
---------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12454967/excessReplicas.1_trunk.txt
  against trunk revision 1038370.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 3 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    -1 release audit.  The applied patch generated 101 release audit warnings (more than the
trunk's current 98 warnings).

    -1 core tests.  The patch failed these core unit tests:
                  org.apache.hadoop.hdfs.server.balancer.TestBalancer
                  org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery
                  org.apache.hadoop.hdfs.server.namenode.TestBlockTokenWithDFS
                  org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
                  org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace
                  org.apache.hadoop.hdfs.server.namenode.TestStorageRestore
                  org.apache.hadoop.hdfs.TestFileConcurrentReader
                  org.apache.hadoop.hdfs.TestLargeBlock

    -1 contrib tests.  The patch failed contrib unit tests.

    +1 system test framework.  The patch passed system test framework compile.

Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/5//testReport/
Release audit warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/5//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/5//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/5//console

This message is automatically generated.

> Exiting safemode takes a long time when there are lots of blocks in the HDFS
> ----------------------------------------------------------------------------
>
>                 Key: HDFS-1391
>                 URL: https://issues.apache.org/jira/browse/HDFS-1391
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: excessReplicas.1_trunk.txt
>
>
> When the namenode decides to exit safemode,  it acquires the FSNamesystem lock and then
iterates over all blocks in the blocksmap to determine if any block has any excess replicas.
This call takes upwards of 5 minutes on a cluster that has 100 million blocks. This delays
namenode restart to a good extent.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message