hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vivek Ratan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4876) Capacity reclaimation for queue would not work as intended
Date Wed, 17 Dec 2008 03:58:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657276#action_12657276
] 

Vivek Ratan commented on HADOOP-4876:
-------------------------------------

To summarize the problem: 

HADOOP-4513 caused some code to be mistakenly removed from TaskSchedulingMgr.updateQSIObject().
This code computed the number of pending tasks, if required, from the list of waiting jobs.
We need to know if there are pending tasks in queue in order to start reclaiming capacity
for the queue if necessary. It's not enough to look at running jobs only. We must start the
timer for reclaiming capacity the moment a job is submitted to a queue that does not have
its Guaranteed Capacity. Consider the following case. You have a queue that is empty, and
its capacity is being utilized by other queues. Now someone submits a job to it. That job
will only show up in the run queue once the initialization thread gets to it, so by looking
at the jobs in the run queue only, you're starting its reclaim timer later than you should.



> Capacity reclaimation for queue would not work as intended
> ----------------------------------------------------------
>
>                 Key: HADOOP-4876
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4876
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>            Reporter: Sreekanth Ramakrishnan
>            Assignee: Sreekanth Ramakrishnan
>            Priority: Critical
>         Attachments: HADOOP-4876-1.patch
>
>
> Capacity schedulers reclaim logic would not work as intended after [HADOOP-4513|https://issues.apache.org/jira/browse/HADOOP-4513]
went in.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message