hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ari Rabkin (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3759) Provide ability to run memory intensive jobs without affecting other running tasks on the nodes
Date Wed, 13 Aug 2008 15:58:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622262#action_12622262
] 

Ari Rabkin commented on HADOOP-3759:
------------------------------------

I had intended to consolidate all the resource estimation and tracking stuff in the ResourceEstimator
class.  Does that seem sensible?

It's possible that some of that stuff belongs, at least in part, in the scheduler.  On the
other hand, tracking current and estimated consumption is in some way orthogonal to all the
other scheduling decisions, so I think it still makes sense to keep resource management in
its own class[es].

> Provide ability to run memory intensive jobs without affecting other running tasks on
the nodes
> -----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3759
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3759
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>             Fix For: 0.19.0
>
>         Attachments: HADOOP-3759.patch, HADOOP-3759.patch, HADOOP-3759.patch
>
>
> In HADOOP-3581, we are discussing how to prevent memory intensive tasks from affecting
Hadoop daemons and other tasks running on a node. A related requirement is that users be provided
an ability to run jobs which are memory intensive. The system must provide enough knobs to
allow such jobs to be run while still maintaining the requirements of HADOOP-3581.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message