hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4085) Kill task attempts longer than a configured queue max time
Date Fri, 30 Mar 2012 15:22:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242427#comment-13242427
] 

Allen Wittenauer commented on MAPREDUCE-4085:
---------------------------------------------

It seems very wrong to me to include this functionality in a specific scheduler.

The other thing to keep in mind is that at least in our use case, we want to punish jobs in
that are essentially in the wrong queue.  Pre-emption works out to be the incorrect action
in this case; we ultimately do want to the job to be failed as negative feedback to the user
that they did something wrong, especially for primarily ad hoc usage (or, as you said, very
heterogeneous).
                
> Kill task attempts longer than a configured queue max time
> ----------------------------------------------------------
>
>                 Key: MAPREDUCE-4085
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4085
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: task
>            Reporter: Allen Wittenauer
>
> For some environments, it is desirable to have certain queues have an SLA with regards
to task turnover.  (i.e., a slot will be free in X minutes and scheduled to the appropriate
job)  Queues should have a 'task time limit' that would cause task attempts over this time
to be killed. This leaves open the possibility that if the task was on a bad node, it could
still be rescheduled up to max.task.attempt times.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message