hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-6673) Add delimited format support to PB OIV tool
Date Wed, 28 Jan 2015 20:52:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14295830#comment-14295830
] 

Haohui Mai edited comment on HDFS-6673 at 1/28/15 8:51 PM:
-----------------------------------------------------------

bq. user can leverage laptop's SSD to get decent performance ...

That's a bold assumption. Even SSD can deliver 50x more IOPS, it still requires 20 hours to
process the fsimage.

bq. In summary, we suggest to use the PB OIV tool as following:

After the offline discussion with [~andrew.wang] and [~eddyxu], the tool is intended to be
a debugging tool for the developers. For this particular purpose I'm willing to change my
-0 to +0. However, as the tool cannot gracefully handle the bigger fsimage in production today,
I think it is necessary to document the applicability for the tool. Let's file a follow up
jira to fix the documentation issue before claiming the tool is ready.


was (Author: wheat9):
bq. user can leverage laptop's SSD to get decent performance ...

That's a bold assumption. Even SSD can deliver 50x more IOPS, it still requires 20 hours to
process the fsimage.

bq. In summary, we suggest to use the PB OIV tool as following:

As pointed out the tool does not handle all existing use cases today. It is insufficient to
just support in the implementation. It is important to document the use cases and make it
explicit on to which extent the tool is applicable.

Please file a follow up jira to fix the documentation issue.

> Add delimited format support to PB OIV tool
> -------------------------------------------
>
>                 Key: HDFS-6673
>                 URL: https://issues.apache.org/jira/browse/HDFS-6673
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: 2.4.0
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
>            Priority: Minor
>             Fix For: 2.7.0
>
>         Attachments: HDFS-6673.000.patch, HDFS-6673.001.patch, HDFS-6673.002.patch, HDFS-6673.003.patch,
HDFS-6673.004.patch, HDFS-6673.005.patch, HDFS-6673.006.patch
>
>
> The new oiv tool, which is designed for Protobuf fsimage, lacks a few features supported
in the old {{oiv}} tool. 
> This task adds supports of _Delimited_ processor to the oiv tool. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message