hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Moved] (HDFS-2251) Namenode does not recognize incorrectly sized blocks
Date Thu, 11 Aug 2011 18:24:27 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eli Collins moved HADOOP-4543 to HDFS-2251:
-------------------------------------------

    Affects Version/s:     (was: 0.18.1)
                  Key: HDFS-2251  (was: HADOOP-4543)
              Project: Hadoop HDFS  (was: Hadoop Common)

> Namenode does not recognize incorrectly sized blocks
> ----------------------------------------------------
>
>                 Key: HDFS-2251
>                 URL: https://issues.apache.org/jira/browse/HDFS-2251
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Brian Bockelman
>
> We had a lot of file system corruption resulting in incorrectly sized blocks (on disk,
they're truncated to 192KB when they should be 64MB).
> However, I cannot make Hadoop realize that these blocks are incorrectly sized.  When
I try to drain off the node, I get the following messages:
> 2008-10-29 18:46:51,293 WARN org.apache.hadoop.fs.FSNamesystem: Inconsistent size for
block blk_-4403534125663454855_9937 reported from 172.16.1.150:50010 current size is 67108864
reported size is 196608
> Here 172.16.1.150 is not the node which has the problematic block, but the destination
of the file transfer.  I propose that Hadoop should either:
> a) Upon startup, make sure that all blocks are properly sized (pro: rather cheap check;
con: doesn't catch any truncations which happen while on disk)
> b) Upon detecting the incorrectly sized copy, Hadoop should ask the source of the block
to perform a block verification.
> Thanks,
> Brian

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message