hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vivek Ratan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4624) CapacityTaskScheduler.MapSchedulingMgr.killTasksFromJob() will not work as expected
Date Wed, 12 Nov 2008 03:23:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12646787#action_12646787

Vivek Ratan commented on HADOOP-4624:

The code does the right thing. It looks for maps that are non-local to any node in the cluster,
then others. It assumes, however, that a job may have some non-local maps (i.e., the JobInProgress
object's _nonLocalRunningMaps_ structure is not empty), as well as other running maps (in
the _runningMapCache_ structure). Amar informs me that these two are mutually exclusive, i.e.,
a job will have one or the other structure empty. So, the right thing to do is modify the
comment in CapacityTaskScheduler.killTasksFromJob() to reflect this, and wrap the calls to
_job.getNonLocalRunningMaps()_ and _job.getRunningMapCache()_ in an if...then...else block.

> CapacityTaskScheduler.MapSchedulingMgr.killTasksFromJob() will not work as expected
> -----------------------------------------------------------------------------------
>                 Key: HADOOP-4624
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4624
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>            Reporter: Amar Kamat
> Once capacity-scheduler decides on killing tasks, it selects running-jobs from the queue
and issues {{killTasksFromJob()}}. The order in which it kills is as follows
> - non-local maps
> - local maps
> _Killing non-local maps :_
> The code here uses {{JobInProgress.getNonLocalRunningMaps()}}. HADOOP-2119 introduced
this for handling cases like _random-writer_. Hence this api will return an empty structure
if there are reducers in the job. Hence the code fails to serve its purpose. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message