hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5752) Provide examples of using offline image viewer (oiv) to analyze hadoop file systems
Date Tue, 28 Apr 2009 00:57:30 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jakob Homan updated HADOOP-5752:

    Attachment: HADOOP-5752.patch

The OIV's output data are ripe for analysis.  The attached patch:
* Creates a new image processor, Delimited, that creates a (by default) tab-delimited file
of the namespace that is suitable for analysis by other tools. 
* Updates the the oiv documentation to provide examples of how to analyze these files using
Pig to find probable duplicate files, files that have never been accessed and the total number
of files of each user in the namespace. These are meant as examples to help ops and such build
other useful scripts.
* Provides unit test for new DelimitedImageVisitor

Right now the script files themselves are not included in the patch because I couldn't figure
out a good place to stash them in the file structure.  Konstantin suggested adding them to
the wiki, which would be nice as other users could add other scripts as they are created,
but I don't see where the wiki hosts files like these.  If it can, can someone please point
me to them?

Santhosh from the Pig team kindly reviewed and blessed the pig scripts.

> Provide examples of using offline image viewer (oiv) to analyze hadoop file systems
> -----------------------------------------------------------------------------------
>                 Key: HADOOP-5752
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5752
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Jakob Homan
>            Assignee: Jakob Homan
>         Attachments: HADOOP-5752.patch
> The offline image viewer provides the ability to generate large amounts of data about
an hdfs namespace. It would be good to provide tools, examples, etc. on how to analyze this
data to find useful information.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message