hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Kling (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-1476) listCorruptFileBlocks should be functional while the name node is still in safe mode
Date Fri, 10 Dec 2010 03:51:02 GMT

     [ https://issues.apache.org/jira/browse/HDFS-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Patrick Kling updated HDFS-1476:
--------------------------------

    Attachment: HDFS-1476.5.patch

- Updated patch to apply to current trunk.
- In BlockManager.markBlockAsCorrupt() only update needed replication queues if they have
been initialized

ant test-patch results:
{code}
     [exec] +1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     +1 tests included.  The patch appears to include 6 new or modified tests.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler
warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
     [exec] 
     [exec]     +1 release audit.  The applied patch does not increase the total number of
release audit warnings.
     [exec] 
     [exec]     +1 system test framework.  The patch passed system test framework compile.
{code}

ant test failures (same as on clean trunk):
{code}
 [junit] Test org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery FAILED
    [junit] Test org.apache.hadoop.hdfs.TestHDFSServerPorts FAILED
    [junit] Test org.apache.hadoop.hdfs.TestHDFSTrash FAILED (timeout)
    [junit] Test org.apache.hadoop.hdfs.server.namenode.TestBackupNode FAILED
    [junit] Test org.apache.hadoop.hdfs.server.namenode.TestStorageRestore FAILED
    [junit] Test org.apache.hadoop.hdfs.TestFileConcurrentReader FAILED (timeout)
    [junit] Test org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete FAILED (timeout)
    [junit] Test org.apache.hadoop.hdfs.server.datanode.TestBlockRecovery FAILED
{code}

> listCorruptFileBlocks should be functional while the name node is still in safe mode
> ------------------------------------------------------------------------------------
>
>                 Key: HDFS-1476
>                 URL: https://issues.apache.org/jira/browse/HDFS-1476
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Patrick Kling
>            Assignee: Patrick Kling
>         Attachments: HDFS-1476.2.patch, HDFS-1476.3.patch, HDFS-1476.4.patch, HDFS-1476.5.patch,
HDFS-1476.patch
>
>
> This would allow us to detect whether missing blocks can be fixed using Raid and if that
is the case exit safe mode earlier.
> One way to make listCorruptFileBlocks available before the name node has exited from
safe mode would be to perform a scan of the blocks map on each call to listCorruptFileBlocks
to determine if there are any blocks with no replicas. This scan could be parallelized by
dividing the space of block IDs into multiple intervals than can be scanned independently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message