hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kang Xiao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4328) Add the option to quiesce the JobTracker
Date Mon, 18 Jun 2012 12:01:45 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13395841#comment-13395841
] 

Kang Xiao commented on MAPREDUCE-4328:
--------------------------------------

It is useful in some condition such as NN is down. Actually we find a way to achieve the first
goal by updating the fair scheduler's conf set each pool's max share to be zero. 
The second goal will protect the job from going to FAILED. But it seems so possible for a
job to go to FAILED since no more task scheduled.

It may be more simple to just not invoke assignTasks() in JobTracker to implement the first
goal. And it will not burden the scheduler implementation since 'safemode' is a small probability
event.
                
> Add the option to quiesce the JobTracker
> ----------------------------------------
>
>                 Key: MAPREDUCE-4328
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4328
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv1
>    Affects Versions: 1.0.3
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>         Attachments: MAPREDUCE-4328.patch
>
>
> In several failure scenarios it would be very handy to have an option to quiesce the
JobTracker.
> Recently, we saw a case where the NameNode had to be rebooted at a customer due to a
random hardware failure - in such a case it would have been nice to not lose jobs by quiescing
the JobTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message