hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arkady Borkovsky <ark...@yahoo-inc.com>
Subject Re: [jira] Commented: (HADOOP-472) Streaming should not crash
Date Tue, 22 Aug 2006 20:27:29 GMT

On Aug 22, 2006, at 11:27 AM, Doug Cutting (JIRA) wrote:

>     [  
> http://issues.apache.org/jira/browse/HADOOP-472? 
> page=comments#action_12429777 ]
>
> Doug Cutting commented on HADOOP-472:
> -------------------------------------
>
> Is it the stack trace that bothers you?  If you want just a single  
> message, then we could replace the stack trace with the exception name  
> and its message.
This is less important than the latter (confusing message).
However, to a user of streaming, a stack trace produced by  
infrastructure signal an error in the infrastructure -- so I'd put  
meaningful messages.


> If you find particular exception messages confusing, then we should  
> fix things so that a more informative exception is thrown, rather than  
> try to translate the messages to something more informative.
Yes, this is more important.
Some can be done by putting more information into exception.
However, certain text may be produced specifically for a human.
E.g.
java.io.FileNotFoundException:  
/export/crawlspace/kryptonite/hadoop/mapred/local/task_0145_r_000000_0/ 
all.2
	at  
org.apache.hadoop.fs.LocalFileSystem.openRaw(LocalFileSystem.java:121)
is less meaningful than:
"it looks like Map step produce no output"

Third:
I believe that Streaming should check the correctness of the task  
before running into exceptions.  (Presence of input files and output  
location, presence of the executables, etc.)
It should also collect the error output of the executables and make it  
available as part of task error output. (Should this be a separate  
Bug?)

AB


>
>> Streaming should not crash
>> --------------------------
>>
>>                 Key: HADOOP-472
>>                 URL: http://issues.apache.org/jira/browse/HADOOP-472
>>             Project: Hadoop
>>          Issue Type: Bug
>>          Components: contrib/streaming
>>            Reporter: arkady borkovsky
>>
>> Streaming framework should not end with a Java stack dump.
>> All the abnormal conditions should be checked and reported.
>> Specific conditions that need to be exmplicitly checked by the  
>> framework are:
>> -- does the input exist?
>> -- do the top level executables or scripts for -mapper and exist
>> Any other Exceptions should also be cought and explained in an error  
>> message.
>
> -- 
> This message is automatically generated by JIRA.
> -
> If you think it was sent incorrectly contact one of the  
> administrators:  
> http://issues.apache.org/jira/secure/Administrators.jspa
> -
> For more information on JIRA, see:  
> http://www.atlassian.com/software/jira
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message