hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1068) In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present
Date Tue, 03 Nov 2009 07:27:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772916#action_12772916
] 

Amareshwari Sriramadasu commented on MAPREDUCE-1068:
----------------------------------------------------

bq. I'm not sure this is the right fix. Would it not be better to print out any exception
caught at line 128?
fail method in streaming is used to fail the job whenever there is an invalid argument, in
which it prints the appropriate message. Users might not want to see an exception trace if
there is an invalid argument.

bq. In working on MAPREDUCE-967 I found that changing that to LOG.fatal("Error in stream job",
ex) printed errors as expected. 
Sorry, I did not understand how MR-967 is related to command-line parsing errors in Streaming.

> In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not
present
> -----------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1068
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1068
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/streaming
>    Affects Versions: 0.20.1
>            Reporter: Peeyush Bishnoi
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>
>         Attachments: patch-1068.txt
>
>
> With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted
and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper
verbose message is thrown if file is not present and it is easy for the users to debug.
> For example:
> With hadoop-0.20.0:
> $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general  -input
/user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer
"/bin/cat"
> Streaming Job Failed!
> With hadoop-0.18.*:
> $ hadoop  jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output
/user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
> 09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy
doesn't exist." found while processing  -file
> ---

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message