hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2573) limit running tasks per job
Date Thu, 10 Jan 2008 17:50:34 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12557724#action_12557724

Doug Cutting commented on HADOOP-2573:

This addresses issues raised in HADOOP-2510.

> limit running tasks per job
> ---------------------------
>                 Key: HADOOP-2573
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2573
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Doug Cutting
>             Fix For: 0.17.0
> It should be possible to specify a limit to the number of tasks per job permitted to
run simultaneously.  If, for example, you have a cluster of 50 nodes, with 100 map task slots
and 100 reduce task slots, and the configured limit is 25 simultaneous tasks/job, then four
or more jobs will be able to run at a time.  This will permit short jobs to pass longer-running
jobs.  This also avoids some problems we've seen with HOD, where nodes are underutilized in
their tail, and it should permit improved input locality.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message