hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4444) Scheduling Information for queues shows one map/reduce task less then actual running map/reduce tasks.
Date Thu, 30 Oct 2008 10:49:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12643807#action_12643807
] 

Vinod K V commented on HADOOP-4444:
-----------------------------------

Found out the actual reason for this behavior. Scheduler obtains running and pending count
from JobInProgress. It does the following when giving out a new task - 
 1) Update queue information. This includes updating running count from all jobs.
 2) Give a new Task. This includes incrementing running count for this job.

The incremented running count for a job in the last step is not reflected in the scheduler
information till the TT reaches scheduler again i.e. when scheduler goes out for giving another
new task to this TT. And when a TT hits the situation when it no longer needs new tasks, running
count incremented in the last assignment doesn't get reflected in the scheduler information
as the TT never reaches scheduler.

A simple fix would be to updated queue information also after task assignment.

> Scheduling Information for queues shows one map/reduce task less then actual running
map/reduce tasks.
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-4444
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4444
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>    Affects Versions: 0.19.0
>         Environment: Hadoop r705159, Queue=default, GC=100% MapCapacity=ReduceCapacity=212
>            Reporter: Karam Singh
>            Assignee: Sreekanth Ramakrishnan
>         Attachments: HADOOP-4444.patch
>
>
> Scheduling Information for queues shows one map/reduce task less then actual running
map/reduce tasks.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message