hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4428) Job Priorities are not handled properly
Date Fri, 17 Oct 2008 15:46:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640547#action_12640547

Vinod K V commented on HADOOP-4428:

My analysis of the cause is correct, but the proposed fix doesn't work as I've learnt that
running jobs shouldn't be preempted no matter what, meaning that running jobs take higher
precedence over waiting jobs irrespective of job priorities.

The correct fix, as I've discussed with Hemanth, is that when running jobs don't have anything
to run, the first job in the waiting queue should be initialized, but nothing can be returned
to the TT as we have to wait for job set-up. TTs that come later should also be returned no
task till the job that is initialized moves to running queue after set-up is done. This way
we make sure that jobs in waiting queue are served in the order of their priority.

I already have a patch for this, working on test-cases. Will put it up on the JIRA asap.

> Job Priorities are not handled properly 
> ----------------------------------------
>                 Key: HADOOP-4428
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4428
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>    Affects Versions: 0.19.0
>         Environment: Cluster:  106 TTs  MapCapacity=212, ReduceCapacity=212
> Single Queue=default, User Limit=25, Priorities = Yes.
> Using hadoop branch 0.19 revision=705159 
>            Reporter: Karam Singh
>            Assignee: Vinod K V
>            Priority: Blocker
>             Fix For: 0.19.0
> Job Priorities are not handled properly 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message