hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vivek Ratan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4624) CapacityTaskScheduler.MapSchedulingMgr.killTasksFromJob() will not work as expected
Date Wed, 12 Nov 2008 03:23:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12646787#action_12646787
] 

Vivek Ratan commented on HADOOP-4624:
-------------------------------------

The code does the right thing. It looks for maps that are non-local to any node in the cluster,
then others. It assumes, however, that a job may have some non-local maps (i.e., the JobInProgress
object's _nonLocalRunningMaps_ structure is not empty), as well as other running maps (in
the _runningMapCache_ structure). Amar informs me that these two are mutually exclusive, i.e.,
a job will have one or the other structure empty. So, the right thing to do is modify the
comment in CapacityTaskScheduler.killTasksFromJob() to reflect this, and wrap the calls to
_job.getNonLocalRunningMaps()_ and _job.getRunningMapCache()_ in an if...then...else block.


> CapacityTaskScheduler.MapSchedulingMgr.killTasksFromJob() will not work as expected
> -----------------------------------------------------------------------------------
>
>                 Key: HADOOP-4624
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4624
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>            Reporter: Amar Kamat
>
> Once capacity-scheduler decides on killing tasks, it selects running-jobs from the queue
and issues {{killTasksFromJob()}}. The order in which it kills is as follows
> - non-local maps
> - local maps
> _Killing non-local maps :_
> The code here uses {{JobInProgress.getNonLocalRunningMaps()}}. HADOOP-2119 introduced
this for handling cases like _random-writer_. Hence this api will return an empty structure
if there are reducers in the job. Hence the code fails to serve its purpose. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message