hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
Date Tue, 03 Nov 2009 07:31:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772918#action_12772918

Todd Lipcon commented on MAPREDUCE-1068:

bq. Users might not want to see an exception trace if there is an invalid argument.

Fair enough for invalid arguments, though I think at least a debug level log with the full
stack trace would be helpful for unit tests.

bq. Sorry, I did not understand how MR-967 is related to command-line parsing errors in Streaming.

It's not - I had to add a unit test for streaming there because the original patch introduced
a regression that didn't already have a test. In working on the streaming unit test, I found
that exceptions thrown inside StreamJob never produced a stack trace in the test logs, making
it pretty difficult to debug.

> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not
> -----------------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-1068
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/streaming
>    Affects Versions: 0.20.1
>            Reporter: Peeyush Bishnoi
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>         Attachments: patch-1068.txt
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted
and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper
verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general  -input
/user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop  jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output
/user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy
doesn't exist." found while processing  -file
> ---

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message