hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-771) Setup and cleanup tasks remain in UNASSIGNED state for a long time on tasktrackers with long running high RAM tasks
Date Mon, 20 Jul 2009 04:28:14 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733121#action_12733121
] 

Hemanth Yamijala commented on MAPREDUCE-771:
--------------------------------------------

test-patch results:

{noformat}
     [exec] -1 overall.
     [exec]
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec]
     [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
     [exec]
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec]
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler
warnings.
     [exec]
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec]
     [exec]     -1 Eclipse classpath. The patch causes the Eclipse classpath to differ from
the contents of the lib directories.
     [exec]
     [exec]     +1 release audit.  The applied patch does not increase the total number of
release audit warnings.
{noformat}

The eclipse classpath problem can be ignored.

> Setup and cleanup tasks remain in UNASSIGNED state for a long time on tasktrackers with
long running high RAM tasks
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-771
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-771
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>    Affects Versions: 0.21.0
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>            Priority: Blocker
>             Fix For: 0.21.0
>
>         Attachments: MAPREDUCE-771.patch, MAPREDUCE-771.patch
>
>
> When a high RAM job's task is scheduled on a tasktracker, the number of slots it occupies
will be more than one. If enough such tasks are scheduled, there can come a situation when
there are no more slots free on the node but the number of tasks running is less than the
number of slots. The jobtracker currently schedules a setup or cleanup task based on how many
tasks are running on the system, rather than slots. As a result of this, it can schedule a
setup or cleanup task on a tasktracker without any free slots. If the high RAM job's tasks
are long running, this will significantly delay the running of the setup or cleanup task,
and thus the entire job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message