hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6673) Add Delimited format supports for PB OIV tool
Date Tue, 20 Jan 2015 23:01:37 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284596#comment-14284596

Haohui Mai commented on HDFS-6673:

bq. Thank you very much to pointing this out. In your patch, you have dumped inodes to LevelDB
sorted by its parent ID. I have tried this method, but in my experiments, the time to dumping
inodes and scan leveldb sequentially overweights the benefits of sequential scanning.

For this particular purpose you don't necessarily store the inode into the db -- putting the
key in the db is sufficient.

bq. I assume that one directory was sequentially written to fsimage. 

This does not hold. FSImage stores the inodes with no order. See {{FSImageFormatPBINode#serializeINodeSection}}.

> Add Delimited format supports for PB OIV tool
> ---------------------------------------------
>                 Key: HDFS-6673
>                 URL: https://issues.apache.org/jira/browse/HDFS-6673
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: 2.4.0
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
>            Priority: Minor
>         Attachments: HDFS-6673.000.patch, HDFS-6673.001.patch, HDFS-6673.002.patch, HDFS-6673.003.patch,
HDFS-6673.004.patch, HDFS-6673.005.patch
> The new oiv tool, which is designed for Protobuf fsimage, lacks a few features supported
in the old {{oiv}} tool. 
> This task adds supports of _Delimited_ processor to the oiv tool. 

This message was sent by Atlassian JIRA

View raw message