hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3759) Provide ability to run memory intensive jobs without affecting other running tasks on the nodes
Date Tue, 15 Jul 2008 17:11:31 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12613663#action_12613663
] 

Hemanth Yamijala commented on HADOOP-3759:
------------------------------------------

One obvious impact is on the scheduling of tasks. If a memory intensive job happens to be
the current job being considered, what would happen if a TT comes in with lesser than required
amount of memory. Scheduling tasks of other jobs to this TT could lead to starvation of the
memory intensive job. Not scheduling other jobs could lead to under utilization of the cluster.
However with different fairness scheduling primitives (like user limits, etc) which are being
discussed in HADOOP-3445 and HADOOP-3746, it might be possible that these situations can be
handled. Some more details will need to be worked out for this aspect.

> Provide ability to run memory intensive jobs without affecting other running tasks on
the nodes
> -----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3759
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3759
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>             Fix For: 0.19.0
>
>
> In HADOOP-3581, we are discussing how to prevent memory intensive tasks from affecting
Hadoop daemons and other tasks running on a node. A related requirement is that users be provided
an ability to run jobs which are memory intensive. The system must provide enough knobs to
allow such jobs to be run while still maintaining the requirements of HADOOP-3581.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message