hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nigel Daley <nda...@yahoo-inc.com>
Subject Re: [jira] Updated: (HADOOP-1304) MAX_TASK_FAILURES should be configurable
Date Tue, 01 May 2007 05:24:52 GMT
Some comments on the JobConf methods:
1) why no setter methods?
2) why are getters getNumMax... instead of getMaxNum...?
3) can this parameters be set per job? per instantiation of the  
JobTracker?
4) the @return javadoc comment should not start with the return type
5) the javadoc should document the default value (IMO)
6) the javadoc should document the property this method relates to
7) remove the word "we" from the javadoc comments -- such as "Get the  
configured number of maximum attempts that will be made to run a map  
task, as specified by the <code>mapred.map.max.attempts</code>  
property. If this property is not already set, the default is 4  
attempts."

On Apr 30, 2007, at 12:47 PM, Devaraj Das (JIRA) wrote:

>
>      [ https://issues.apache.org/jira/browse/HADOOP-1304? 
> page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Devaraj Das updated HADOOP-1304:
> --------------------------------
>
>     Attachment: 1304.patch
>
> Attached is another patch with the JobConf accessor methods.
> Regarding the potential problem that Arun raised, yes, the problem  
> exists but the hope is that HADOOP-785 will address these  
> vulnerabilities.
>
>> MAX_TASK_FAILURES should be configurable
>> ----------------------------------------
>>
>>                 Key: HADOOP-1304
>>                 URL: https://issues.apache.org/jira/browse/ 
>> HADOOP-1304
>>             Project: Hadoop
>>          Issue Type: Improvement
>>          Components: mapred
>>    Affects Versions: 0.12.3
>>            Reporter: Christian Kunz
>>         Assigned To: Devaraj Das
>>         Attachments: 1304.patch, 1304.patch, 1304.patch
>>
>>
>> After a couple of weeks of failed attempts I was able to finish a  
>> large job only after I changed MAX_TASK_FAILURES to a higher  
>> value. In light of HADOOP-1144 (allowing a certain amount of task  
>> failures without failing the job) it would be even better if this  
>> value could be configured separately for mappers and reducers,  
>> because often a success of a job requires the success of all  
>> reducers but not of all mappers.
>
> -- 
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>


Mime
View raw message