hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinay (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5723) Append failed FINALIZED replica should not be accepted as valid when that block is underconstruction
Date Fri, 24 Jan 2014 05:10:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880721#comment-13880721
] 

Vinay commented on HDFS-5723:
-----------------------------

bq. Hi Vinay, one question about the patch: so this inconsistent generation stamp can also
be caused by a delayed block-received report? I.e., after the first close(), the DN's report
gets delayed and is received by NN when the append starts. In that case, will we have any
issue by wrongly putting the (block, DN) into the corruptBlockMap?
It can happen. Earlier block with prev genstamp will be marked corrupt, if append pipeline
setup with genstamp update happened before previous report comes.
But append pipeline creation will update the genstamp in that datanode also, so one more block-received
report is expectedn with correct genstamp. This time it will remove the old block from corrupt
replica map.

> Append failed FINALIZED replica should not be accepted as valid when that block is underconstruction
> ----------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-5723
>                 URL: https://issues.apache.org/jira/browse/HDFS-5723
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.2.0
>            Reporter: Vinay
>            Assignee: Vinay
>         Attachments: HDFS-5723.patch
>
>
> Scenario:
> 1. 3 node cluster with dfs.client.block.write.replace-datanode-on-failure.enable set
to false.
> 2. One file is written with 3 replicas, blk_id_gs1
> 3. One of the datanode DN1 is down.
> 4. File was opened with append and some more data is added to the file and synced. (to
only 2 live nodes DN2 and DN3)-- blk_id_gs2
> 5. Now  DN1 restarted
> 6. In this block report, DN1 reported FINALIZED block blk_id_gs1, this should be marked
corrupted.
> but since NN having appended block state as UnderConstruction, at this time its not detecting
this block as corrupt and adding to valid block locations.
> As long as the namenode is alive, this datanode also will be considered as valid replica
and read/append will fail in that datanode.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message