hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5723) Append failed FINALIZED replica should not be accepted as valid when that block is underconstruction
Date Wed, 23 Jul 2014 15:14:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14071844#comment-14071844
] 

Uma Maheswara Rao G commented on HDFS-5723:
-------------------------------------------

For the above question which Jing asked should work fine as Vinay answered it.

When DN reports with older genstamp, NN will add it to corruptReplicMap and NN ask DN to remove
block with older genstamp only when it processes corrupt replicas. DN can not remove it as
DN already bumped with genstamp. on next report from DN with newer genstamp, NN will remove
from corruptMap if it exist.

{code}
 } else {
      // if the same block is added again and the replica was corrupt
      // previously because of a wrong gen stamp, remove it from the
      // corrupt block list.
      corruptReplicas.removeFromCorruptReplicasMap(block, node,
          Reason.GENSTAMP_MISMATCH);
      curReplicaDelta = 0;
      blockLog.warn("BLOCK* addStoredBlock: "
          + "Redundant addStoredBlock request received for " + storedBlock
          + " on " + node + " size " + storedBlock.getNumBytes());
    }
{code}



> Append failed FINALIZED replica should not be accepted as valid when that block is underconstruction
> ----------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-5723
>                 URL: https://issues.apache.org/jira/browse/HDFS-5723
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.2.0
>            Reporter: Vinayakumar B
>            Assignee: Vinayakumar B
>         Attachments: HDFS-5723.patch, HDFS-5723.patch, HDFS-5723.patch
>
>
> Scenario:
> 1. 3 node cluster with dfs.client.block.write.replace-datanode-on-failure.enable set
to false.
> 2. One file is written with 3 replicas, blk_id_gs1
> 3. One of the datanode DN1 is down.
> 4. File was opened with append and some more data is added to the file and synced. (to
only 2 live nodes DN2 and DN3)-- blk_id_gs2
> 5. Now  DN1 restarted
> 6. In this block report, DN1 reported FINALIZED block blk_id_gs1, this should be marked
corrupted.
> but since NN having appended block state as UnderConstruction, at this time its not detecting
this block as corrupt and adding to valid block locations.
> As long as the namenode is alive, this datanode also will be considered as valid replica
and read/append will fail in that datanode.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message