hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-569) Hadoop should allow the user to dynamically change the number of times to re-try failed tasks before declaring the job fail
Date Wed, 04 Oct 2006 04:19:20 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-569?page=comments#action_12439720 ] 
            
Owen O'Malley commented on HADOOP-569:
--------------------------------------

One part that should be changed before this is considered is to remove the generation of all
of the possible task names for a job at job creation. With 4 tasks/tip, it is lame, but not
that harmful. If someone set the number of tasks/tip to 10000, it would be a big problem.


Additionally, changing the number dynamically is a big jump from the current model. It would
be much more consistent to allow the user to set it in the JobConf at job submission time.
What is the advantage of changing it dynamically?


> Hadoop should allow the user to dynamically change the number of times to re-try failed
tasks before declaring the job fail
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-569
>                 URL: http://issues.apache.org/jira/browse/HADOOP-569
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Runping Qi
>
> Hadoop has a built-in mechanism to fail a job if some tasks failed more than 3 times.
This mechanism works fine in most scenarios. However, in some other cases, it is highly desirable
for the user to change (increase) that number. My current running job demonstrates such a
scenario: The job has run more than 2.5 days. It is close to complete (90+%). Everything indicates
that it will finish eventually in a day, except for one potential danger: some of the tasks
are in their 3rd try! 
> It will be extremely helpful if I can change the maximun number of tries to 6 instead
of 4!

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message