Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5219018FCE for ; Tue, 27 Oct 2015 17:41:50 +0000 (UTC) Received: (qmail 64916 invoked by uid 500); 27 Oct 2015 17:41:28 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 64848 invoked by uid 500); 27 Oct 2015 17:41:28 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 64799 invoked by uid 99); 27 Oct 2015 17:41:28 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Oct 2015 17:41:28 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id CFFF22C1F62 for ; Tue, 27 Oct 2015 17:41:27 +0000 (UTC) Date: Tue, 27 Oct 2015 17:41:27 +0000 (UTC) From: "Chang Li (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HDFS-9289) check genStamp when complete file MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-9289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976815#comment-14976815 ] Chang Li commented on HDFS-9289: -------------------------------- [~zhz], I don't have the log show the file was completed with an old GS. But by look up the block from jsp page right now, I can see that the block blk_3773617405 currently has replica on host ***657n26.***.com, ***656n04.***.com, and ***656n38.***.com, by going to those datanode, I see the replica on those datanodes have replica with old genstamp. {code} bash-4.1$ hostname ***657n26.***.com bash-4.1$ ls -l /grid/2/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405* -rw-r--r-- 1 hdfs users 107761275 Oct 23 18:00 /grid/2/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405 -rw-r--r-- 1 hdfs users 841895 Oct 23 18:00 /grid/2/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405_1106111498065.meta bash-4.1$ hostname ***656n04.***.com bash-4.1$ ls -l /grid/1/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405* -rw-r--r-- 1 hdfs users 107761275 Oct 21 19:14 /grid/1/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405 -rw-r--r-- 1 hdfs users 841895 Oct 21 19:14 /grid/1/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405_1106111498065.meta bash-4.1$ hostname ***656n38.***.com bash-4.1$ ls -l /grid/3/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405* -rw-r--r-- 1 hdfs users 107761275 Oct 23 09:14 /grid/3/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405 -rw-r--r-- 1 hdfs users 841895 Oct 23 09:14 /grid/3/hadoop/var/hdfs/data/current/BP-1052427332-98.138.108.146-1350583571998/current/finalized/subdir236/subdir212/blk_3773617405_1106111498065.meta {code} > check genStamp when complete file > --------------------------------- > > Key: HDFS-9289 > URL: https://issues.apache.org/jira/browse/HDFS-9289 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: Chang Li > Assignee: Chang Li > Priority: Critical > Attachments: HDFS-9289.1.patch, HDFS-9289.2.patch > > > we have seen a case of corrupt block which is caused by file complete after a pipelineUpdate, but the file complete with the old block genStamp. This caused the replicas of two datanodes in updated pipeline to be viewed as corrupte. Propose to check genstamp when commit block -- This message was sent by Atlassian JIRA (v6.3.4#6332)