hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HADOOP-5133) FSNameSystem#addStoredBlock does not handle inconsistent block length correctly
Date Wed, 11 Feb 2009 01:19:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672476#action_12672476
] 

hairong edited comment on HADOOP-5133 at 2/10/09 5:18 PM:
----------------------------------------------------------------

The logic in summary 3 should only apply to the last block of a file under construction. A
complete picture should be:
{noformat}
if the reported block is the last block of an under-construction file {
     do summary 3
} else {
    if the reported block is not the last block && its NN recorded length is not equal
to the preferred size
         set the block's NN recorded length to be the preferred block size;
    if the reported length is not equal to the NN recorded length
         mark the reported block as corrupt;
}
{noformat}

      was (Author: hairong):
    The logic in summary 3 should only apply to the last block of a file under construction.
A complete picture should be:
if the reported block is the last block of an under-construction file {
     do summary 3
} else {
    if the reported block is not the last block of a file and NN recorded length is not equal
to the preferred block length, set the block's NN recorded length to be the preferred block
size;
    if the reported length is not equal to the NN recorded length, mark the reported block
as corrupt;
}
  
> FSNameSystem#addStoredBlock does not handle inconsistent block length correctly
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-5133
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5133
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.18.2
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>            Priority: Blocker
>             Fix For: 0.18.4
>
>         Attachments: inconsistentLen.patch
>
>
> Currently NameNode treats either the new replica or existing replicas as corrupt if the
new replica's length is inconsistent with NN recorded block length. The correct behavior should
be
> 1. For a block that is not under construction, the new replica should be marked as corrupt
if its length is inconsistent (no matter shorter or longer) with the NN recorded block length;
> 2. For an under construction block, if the new replica's length is shorter than the NN
recorded block length, the new replica could be marked as corrupt; if the new replica's length
is longer, NN should update its recorded block length. But it should not mark existing replicas
as corrupt. This is because NN recorded length for an under construction block does not accurately
match the block length on datanode disk. NN should not judge an under construction replica
to be corrupt by looking at the inaccurate information:  its recorded block length.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message