Return-Path: <hdfs-issues-return-171898-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id 89BE5200BA6
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Tue, 18 Oct 2016 22:06:00 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id 8827B160ACE; Tue, 18 Oct 2016 20:06:00 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id CB10F160AE5
	for <archive-asf-public@cust-asf.ponee.io>; Tue, 18 Oct 2016 22:05:59 +0200 (CEST)
Received: (qmail 3688 invoked by uid 500); 18 Oct 2016 20:05:59 -0000
Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:hdfs-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:hdfs-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:hdfs-issues@hadoop.apache.org>
List-Id: <hdfs-issues.hadoop.apache.org>
Delivered-To: mailing list hdfs-issues@hadoop.apache.org
Received: (qmail 3648 invoked by uid 99); 18 Oct 2016 20:05:58 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Oct 2016 20:05:58 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 806752C4C7B
	for <hdfs-issues@hadoop.apache.org>; Tue, 18 Oct 2016 20:05:58 +0000 (UTC)
Date: Tue, 18 Oct 2016 20:05:58 +0000 (UTC)
From: "Xiao Chen (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13012212.1476407061000.20656.1476821158522@Atlassian.JIRA>
In-Reply-To: <JIRA.13012212.1476407061000@Atlassian.JIRA>
References: <JIRA.13012212.1476407061000@Atlassian.JIRA> <JIRA.13012212.1476407061057@arcas>
Subject: [jira] [Comment Edited] (HDFS-11009) Add a tool to reconstruct
 block meta file from CLI
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
archived-at: Tue, 18 Oct 2016 20:06:00 -0000


    [ https://issues.apache.org/jira/browse/HDFS-11009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586524#comment-15586524 ] 

Xiao Chen edited comment on HDFS-11009 at 10/18/16 8:05 PM:
------------------------------------------------------------

Thanks Andrew for explaining and more reviews.

I see your point now... added more text to make this scarier and more accurate. How does it feel now? The 'if' text is also fixed

CLI output looks like this now:
{noformat}
xiao$ bin/hdfs debug computeMeta
computeMeta -block <block-file> -out <output-metadata-file>
  Compute HDFS metadata from the specified block file, and save it to
  the specified output metadata file.

**NOTE: Use at your own risk!
 If the block file is corrupt and you overwrite it's meta file, 
 it will show up as good in HDFS, but you can't read the data.
 Only use as a last measure, and when you are 100% certain the block file is good.
{noformat}
The assumption here is that when people is running the {{debug}} command, they know what they're doing. (At least need to figure out which block file from which HDFS file is of interest, and where that block file locates etc.)

bq. Could we fix the verifyMeta help text while we're here to also remove the "[]"? The "-block" flag looks required to me.
Not really, so if you {{verify}} a meta file, it checks it can be read as a meta. If you also give a block file, it then verifies checksum matches. :)


was (Author: xiaochen):
Thanks Andrew for explaining and more reviews.

I see your point now... added more text to make this scarier and more accurate. How does it feel now? The 'if' text is also fixed
cmd line output looks like this:
{noformat}
xiao$ bin/hdfs debug computeMeta
computeMeta -block <block-file> -out <output-metadata-file>
  Compute HDFS metadata from the specified block file, and save it to
  the specified output metadata file.

**NOTE: Use at your own risk!
 If the block file is corrupt and you overwrite it's meta file, 
 it will show up as good in HDFS, but you can't read the data.
 Only use as a last measure, and when you are 100% certain the block file is good.
{noformat}
The assumption here is that when people is running the {{debug}} command, they know what they're doing. (At least need to figure out which block file from which HDFS file is of interest, and where that block file locates etc.)

bq. Could we fix the verifyMeta help text while we're here to also remove the "[]"? The "-block" flag looks required to me.
Not really, so if you {{verify}} a meta file, it checks it can be read as a meta. If you also give a block file, it then verifies checksum matches. :)

> Add a tool to reconstruct block meta file from CLI
> --------------------------------------------------
>
>                 Key: HDFS-11009
>                 URL: https://issues.apache.org/jira/browse/HDFS-11009
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Xiao Chen
>            Assignee: Xiao Chen
>         Attachments: HDFS-11009.01.patch, HDFS-11009.02.patch, HDFS-11009.03.patch, HDFS-11009.04.patch
>
>
> If the block file on local disk presents, but the meta file is missing, it's theoretically possible to manually restore the meta file and have the block readable.
> This jira is to propose to add such a tool to do that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org