hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HADOOP-4810) Data lost at cluster startup time
Date Tue, 09 Dec 2008 20:21:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654943#action_12654943
] 

hairong edited comment on HADOOP-4810 at 12/9/08 12:21 PM:
-----------------------------------------------------------------

The data got lost when we upgraded our cluster from 0.17 to 0.18. The root cause of the problem
is HADOOP-4663 which introduced a lot of corrupted blocks into dfs and thus polluted the data.

      was (Author: hairong):
    The data got lost when we upgraded our cluster from 0.17 to 0.18. The root cause of the
problem is HADOOP-4702 which introduced a lot of corrupted blocks into dfs and thus polluted
the data.
  
> Data lost at cluster startup time
> ---------------------------------
>
>                 Key: HADOOP-4810
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4810
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.18.2
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>            Priority: Blocker
>             Fix For: 0.18.3
>
>
> hadoop dfs -cat file1 returns
> dfs.DFSClient: Could not obtain block blk_XX_0 from any node: java.io.IOException: No
live nodes contain current block
> Tracing the history of the block from NN log, we found
>  WARN org.apache.hadoop.fs.FSNamesystem: Inconsistent size for block blk_-6160940519231606858_0
reported from A1.A2.A3.A4:50010 current size is 9303872 reported size is 262144
>  WARN org.apache.hadoop.fs.FSNamesystem: Deleting block blk_-6160940519231606858_0 from
A1.A2.A3.A4:50010
> INFO org.apache.hadoop.dfs.StateChange: DIR* NameSystem.invalidateBlock: blk_-6160940519231606858_0
on A1.A2.A3.A4:50010 
> WARN org.apache.hadoop.fs.FSNamesystem: Error in deleting bad block blk_-6160940519231606858_0
org.apache.hadoop.dfs.SafeModeException: Cannot invalidate block blk_-6160940519231606858_0.
Name node is in safe mode. 
> WARN org.apache.hadoop.fs.FSNamesystem: Inconsistent size for block blk_-6160940519231606858_0
reported from B1.B2.B3.B4:50010 current size is 9303872 reported size is 306688 
> WARN org.apache.hadoop.fs.FSNamesystem: Deleting block blk_-6160940519231606858_0 from
B1.B2.B3.B4:50010 
> INFO org.apache.hadoop.dfs.StateChange: DIR* NameSystem.invalidateBlock: blk_-6160940519231606858_0
on B1.B2.B3.B4:50010 
> WARN org.apache.hadoop.fs.FSNamesystem: Error in deleting bad block blk_-6160940519231606858_0
org.apache.hadoop.dfs.SafeModeException: Cannot invalidate block blk_-6160940519231606858_0.
Name node is in safe mode. 
> INFO org.apache.hadoop.dfs.StateChange: BLOCK* NameSystem.chooseExcessReplicates: (C1.C2.C3.C4:50010,
blk_-6160940519231606858_0) is added to recentInvalidateSets 
> INFO org.apache.hadoop.dfs.StateChange: BLOCK* NameSystem.chooseExcessReplicates: (D1.D2.D3.D4:50010,
blk_-6160940519231606858_0) is added to recentInvalidateSets
> INFO org.apache.hadoop.dfs.StateChange: BLOCK* ask C1.C2.C3.C4:50010 to delete blk_-6160940519231606858_0
> INFO org.apache.hadoop.dfs.StateChange: BLOCK* ask D1.D2.D3.D4:50010 to delete blk_-6160940519231606858_0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message