hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Gummadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-2078) TraceBuilder unable to generate the traces while giving the job history path by globing.
Date Tue, 21 Sep 2010 08:32:34 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12912877#action_12912877
] 

Ravi Gummadi commented on MAPREDUCE-2078:
-----------------------------------------

Some comments on the patch:
(1) I think FileSystem.createNewFile(inputPath1) is better/simpler than FsShell.run(new String[]
{"-touchz", inputPath1.toString()}) in the testcase.
(2) The method name processInput() can be changed to something like processInputArgument()
or buildInputHistoryPaths() for more clarity.
(3) In FileInputFormat.listStatus(), the return value of fs.globStatus() is checked for null.
Please check if we need the check here also.

> TraceBuilder unable to generate the traces while giving the job history path by globing.
> ----------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2078
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2078
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tools/rumen
>            Reporter: Vinay Kumar Thota
>            Assignee: Amar Kamat
>         Attachments: mapreduce-2078-v1.2.patch
>
>
> I was trying to generate the traces for MR job histories by using TraceBuilder. However,
it's unable to generate the traces while giving the job history path by globing. It throws
a file not found exception even though the job history path is exists.
> I have provide the job history path in the below way.
> hdfs://<<clustername>>/dir1/dir2/dir3/*/*/*/*/*/*/
> Exception:
> java.io.FileNotFoundException: File does not exist:
> hdfs://<<clustername>>/dir1/dir2/dir3/*/*/*/*/*/*
>         at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:525)
>         at org.apache.hadoop.tools.rumen.TraceBuilder$MyOptions.<init>(TraceBuilder.java:88)
>         at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:183)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>         at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:121)
> It's truncating the last  slash in the path.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message