hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4623) Running tasks are not maintained by JobInProgress if speculation is off
Date Wed, 26 Nov 2008 13:04:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amar Kamat updated HADOOP-4623:
-------------------------------

    Attachment: HADOOP-4623-v1.3.patch

Attaching a patch the incorporates Amareshwari's comment #1. I have added a testcase that
requires {{WaitingMapper}} and the related _utils_ and hence I have re-factored the code.
Here are the benchmark results
||trunk(secs)||trunk+4623-v1.2.patch(secs)||diff(secs)||
|2440|2407|-33|
| |2432|-8 secs|
| |2434|-6 secs|

> Running tasks are not maintained by JobInProgress if speculation is off
> -----------------------------------------------------------------------
>
>                 Key: HADOOP-4623
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4623
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>         Attachments: HADOOP-4623-v1.1.patch, HADOOP-4623-v1.2.patch, HADOOP-4623-v1.3.patch
>
>
> {{JobInProgress}} doesnt maintain any structure for running tasks if speculation is turned
_off_.  {{getRunningMapCache()}} in {{JobInProgress}} exposes the running map cache. This
api returns an empty {{Map}} if speculation turned off. 
> _Usage_ :
> {{CapicityScheduler}} requires a list of running tasks for both speculated and non-speculated
jobs. See HADOOP-4558 to see how this issue affects {{CapacityScheduler}}.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message