hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1448) Create multi-format parser for edits logs file, support binary and XML formats initially
Date Fri, 10 Dec 2010 01:07:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970037#action_12970037
] 

Jakob Homan commented on HDFS-1448:
-----------------------------------

Looking good.  Thanks for the changes.

Review of latest patch (line numbers refer to patch):

* 544: typo: judgement -> judgment
* 785: Commented out code
* 822-824: The call to the MiniDFSCluster builder can be simplified and the builder variable
eliminated.
* 3045: Usage should show option of using long option, not just -i/-o.
* 3045, cont.: Usage should also document other command line options and statistics viewer.
* 3045ish: For specifying the processor, we should probably follow the oiv convention of -p
* 3867: The call to logEdit is broken in a bit of an odd place.  Does the call not fit on
one line?
* FSEditLog: Would the code be cleaner with static imports of the FSEDitLogOpCodes?
* The oev will need comparable documentation to what is provided for the oiv.


> Create multi-format parser for edits logs file, support binary and XML formats initially
> ----------------------------------------------------------------------------------------
>
>                 Key: HDFS-1448
>                 URL: https://issues.apache.org/jira/browse/HDFS-1448
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: tools
>    Affects Versions: 0.22.0
>            Reporter: Erik Steffl
>            Assignee: Erik Steffl
>             Fix For: 0.22.0
>
>         Attachments: editsStored, HDFS-1448-0.22-1.patch, HDFS-1448-0.22-2.patch, HDFS-1448-0.22-3.patch,
HDFS-1448-0.22.patch, Viewer hierarchy.pdf
>
>
> Create multi-format parser for edits logs file, support binary and XML formats initially.
> Parsing should work from any supported format to any other supported format (e.g. from
binary to XML and from XML to binary).
> The binary format is the format used by FSEditLog class to read/write edits file.
> Primary reason to develop this tool is to help with troubleshooting, the binary format
is hard to read and edit (for human troubleshooters).
> Longer term it could be used to clean up and minimize parsers for fsimage and edits files.
Edits parser OfflineEditsViewer is written in a very similar fashion to OfflineImageViewer.
Next step would be to merge OfflineImageViewer and OfflineEditsViewer and use the result in
both FSImage and FSEditLog. This is subject to change, specifically depending on adoption
of avro (which would completely change how objects are serialized as well as provide ways
to convert files to different formats).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message