hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chang Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9289) check genStamp when complete file
Date Tue, 27 Oct 2015 17:41:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976815#comment-14976815
] 

Chang Li commented on HDFS-9289:
--------------------------------

[~zhz], I don't have the log show the file was completed with an old GS. But by look up the
block from jsp page right now, I can see that the block blk_3773617405 currently has replica
on host ***657n26.***.com, ***656n04.***.com, and ***656n38.***.com,
by going to those datanode, I see the replica on those datanodes have replica with old genstamp.
{code}
bash-4.1$ hostname
***657n26.***.com
bash-4.1$ ls -l /grid/2/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405*
-rw-r--r-- 1 hdfs users 107761275 Oct 23 18:00 /grid/2/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405
-rw-r--r-- 1 hdfs users    841895 Oct 23 18:00 /grid/2/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405_1106111498065.meta

bash-4.1$ hostname
***656n04.***.com
bash-4.1$ ls -l /grid/1/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405*
-rw-r--r-- 1 hdfs users 107761275 Oct 21 19:14 /grid/1/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405
-rw-r--r-- 1 hdfs users    841895 Oct 21 19:14 /grid/1/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405_1106111498065.meta

bash-4.1$ hostname
***656n38.***.com
bash-4.1$ ls -l /grid/3/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405*
-rw-r--r-- 1 hdfs users 107761275 Oct 23 09:14 /grid/3/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405
-rw-r--r-- 1 hdfs users    841895 Oct 23 09:14 /grid/3/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405_1106111498065.meta
{code}

> check genStamp when complete file
> ---------------------------------
>
>                 Key: HDFS-9289
>                 URL: https://issues.apache.org/jira/browse/HDFS-9289
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Chang Li
>            Assignee: Chang Li
>            Priority: Critical
>         Attachments: HDFS-9289.1.patch, HDFS-9289.2.patch
>
>
> we have seen a case of corrupt block which is caused by file complete after a pipelineUpdate,
but the file complete with the old block genStamp. This caused the replicas of two datanodes
in updated pipeline to be viewed as corrupte. Propose to check genstamp when commit block



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message