On second thoughts if we plan to support file patterns for input
paths both in streaming and general map reduce. Then either we should
be match and see if some files matching the pattern exist in DFS or
not check at all. Otherwise this check will break functionality.
Sanjay
On 30-Nov-06, at 7:23 PM, Sanjay Dahiya wrote:
> It does take care of existence input in DFS, scripts on local FS
> and read permission for scripts. Execute permission check on a file
> is coming in java 6, so for now I have chmod +x on the file after
> checking other permissions.
> On descriptive error messages there is another JIRA on that
> HADOOP-676, that patch contains some changes to improve errors and
> throwing new exceptions.
>
> Sanjay
>
> On 30-Nov-06, at 1:46 AM, Arkady Borkovsky wrote:
>
>> Sanjay,
>>
>> does you implementation that ALL the scripts and executable
>> available and have right permissions, and that the input exists?
>> Also: what about descriptive error messages?
>>
>> -- ab
>>
>> On Nov 29, 2006, at 7:43 AM, Sanjay Dahiya (JIRA) wrote:
>>
>>> [ http://issues.apache.org/jira/browse/HADOOP-476?
>>> page=comments#action_12454343 ]
>>>
>>> Sanjay Dahiya commented on HADOOP-476:
>>> --------------------------------------
>>>
>>> I have the cli2 implementation in place, there is one issue with
>>> cli2 which is holding it back. Command line arguments with -
>>> jobconf and -env get merged in the same properties set in CLI's
>>> parser and after parsing we dont know what property was passed
>>> with which trigger.
>>> I have raised on CLI jira CLI-126. I'll submit the patch once i
>>> resolve this one..
>>>
>>> (http://issues.apache.org/jira/browse/CLI-126) in case auto link
>>> doesnt work for other projects.
>>>
>>>> Streaming should check for correctness of the task
>>>> --------------------------------------------------
>>>>
>>>> Key: HADOOP-476
>>>> URL: http://issues.apache.org/jira/browse/
>>>> HADOOP-476
>>>> Project: Hadoop
>>>> Issue Type: Bug
>>>> Components: contrib/streaming
>>>> Reporter: arkady borkovsky
>>>> Assigned To: Sanjay Dahiya
>>>>
>>>> Currently, if anythin is wrong with streaming job, it dies
>>>> without any explanation.
>>>> Before creating and running actual MapReduce job, Streaming
>>>> should check if:
>>>> -- the executables (or scripts) for -mapper and -reducer are
>>>> available and have right permissions
>>>> -- the input fragments exist
>>>
>>> --
>>> This message is automatically generated by JIRA.
>>> -
>>> If you think it was sent incorrectly contact one of the
>>> administrators: http://issues.apache.org/jira/secure/
>>> Administrators.jspa
>>> -
>>> For more information on JIRA, see: http://www.atlassian.com/
>>> software/jira
>>>
>>>
>>
>
|