hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11009) Add a tool to reconstruct block meta file from CLI
Date Tue, 18 Oct 2016 20:05:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586524#comment-15586524
] 

Xiao Chen edited comment on HDFS-11009 at 10/18/16 8:05 PM:
------------------------------------------------------------

Thanks Andrew for explaining and more reviews.

I see your point now... added more text to make this scarier and more accurate. How does it
feel now? The 'if' text is also fixed

CLI output looks like this now:
{noformat}
xiao$ bin/hdfs debug computeMeta
computeMeta -block <block-file> -out <output-metadata-file>
  Compute HDFS metadata from the specified block file, and save it to
  the specified output metadata file.

**NOTE: Use at your own risk!
 If the block file is corrupt and you overwrite it's meta file, 
 it will show up as good in HDFS, but you can't read the data.
 Only use as a last measure, and when you are 100% certain the block file is good.
{noformat}
The assumption here is that when people is running the {{debug}} command, they know what they're
doing. (At least need to figure out which block file from which HDFS file is of interest,
and where that block file locates etc.)

bq. Could we fix the verifyMeta help text while we're here to also remove the "[]"? The "-block"
flag looks required to me.
Not really, so if you {{verify}} a meta file, it checks it can be read as a meta. If you also
give a block file, it then verifies checksum matches. :)


was (Author: xiaochen):
Thanks Andrew for explaining and more reviews.

I see your point now... added more text to make this scarier and more accurate. How does it
feel now? The 'if' text is also fixed
cmd line output looks like this:
{noformat}
xiao$ bin/hdfs debug computeMeta
computeMeta -block <block-file> -out <output-metadata-file>
  Compute HDFS metadata from the specified block file, and save it to
  the specified output metadata file.

**NOTE: Use at your own risk!
 If the block file is corrupt and you overwrite it's meta file, 
 it will show up as good in HDFS, but you can't read the data.
 Only use as a last measure, and when you are 100% certain the block file is good.
{noformat}
The assumption here is that when people is running the {{debug}} command, they know what they're
doing. (At least need to figure out which block file from which HDFS file is of interest,
and where that block file locates etc.)

bq. Could we fix the verifyMeta help text while we're here to also remove the "[]"? The "-block"
flag looks required to me.
Not really, so if you {{verify}} a meta file, it checks it can be read as a meta. If you also
give a block file, it then verifies checksum matches. :)

> Add a tool to reconstruct block meta file from CLI
> --------------------------------------------------
>
>                 Key: HDFS-11009
>                 URL: https://issues.apache.org/jira/browse/HDFS-11009
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Xiao Chen
>            Assignee: Xiao Chen
>         Attachments: HDFS-11009.01.patch, HDFS-11009.02.patch, HDFS-11009.03.patch, HDFS-11009.04.patch
>
>
> If the block file on local disk presents, but the meta file is missing, it's theoretically
possible to manually restore the meta file and have the block readable.
> This jira is to propose to add such a tool to do that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message