hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjay Dahiya <sanj...@yahoo-inc.com>
Subject Re: [jira] Commented: (HADOOP-476) Streaming should check for correctness of the task
Date Thu, 30 Nov 2006 13:53:50 GMT
It does take care of existence input in DFS,  scripts on local FS and  
read permission for scripts. Execute permission check on a file is  
coming in java 6, so for now I have chmod +x on the file after  
checking other permissions.
On descriptive error messages there is another JIRA on that  
HADOOP-676, that patch contains some changes to improve errors and  
throwing new exceptions.

Sanjay

On 30-Nov-06, at 1:46 AM, Arkady Borkovsky wrote:

> Sanjay,
>
> does you implementation that ALL the scripts and executable  
> available and have right permissions, and that the input exists?   
> Also: what about descriptive error messages?
>
> -- ab
>
> On Nov 29, 2006, at 7:43 AM, Sanjay Dahiya (JIRA) wrote:
>
>>     [ http://issues.apache.org/jira/browse/HADOOP-476? 
>> page=comments#action_12454343 ]
>>
>> Sanjay Dahiya commented on HADOOP-476:
>> --------------------------------------
>>
>> I have the cli2 implementation in place, there is one issue with  
>> cli2 which is holding it back. Command line arguments with - 
>> jobconf and -env get merged in the same properties set in CLI's  
>> parser and after parsing we dont know what property was passed  
>> with which trigger.
>> I have raised on CLI jira CLI-126. I'll submit the patch once i  
>> resolve this one..
>>
>> (http://issues.apache.org/jira/browse/CLI-126) in case auto link  
>> doesnt work for other projects.
>>
>>> Streaming should check for correctness of the task
>>> --------------------------------------------------
>>>
>>>                 Key: HADOOP-476
>>>                 URL: http://issues.apache.org/jira/browse/HADOOP-476
>>>             Project: Hadoop
>>>          Issue Type: Bug
>>>          Components: contrib/streaming
>>>            Reporter: arkady borkovsky
>>>         Assigned To: Sanjay Dahiya
>>>
>>> Currently, if anythin is wrong with streaming job, it dies  
>>> without any explanation.
>>> Before creating and running actual MapReduce job, Streaming  
>>> should check if:
>>> -- the executables (or scripts) for -mapper and -reducer are  
>>> available and have right permissions
>>> -- the input fragments exist
>>
>> -- 
>> This message is automatically generated by JIRA.
>> -
>> If you think it was sent incorrectly contact one of the  
>> administrators: http://issues.apache.org/jira/secure/ 
>> Administrators.jspa
>> -
>> For more information on JIRA, see: http://www.atlassian.com/ 
>> software/jira
>>
>>
>


Mime
View raw message