Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <103618790.1218648044865.JavaMail.jira@brutus>
Date: Wed, 13 Aug 2008 10:20:44 -0700 (PDT)
From: "Hemanth Yamijala (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-3759) Provide ability to run memory
 intensive jobs without affecting other running tasks on the nodes
In-Reply-To: <1898755466.1216109611599.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622284#action_12622284 ] 

Hemanth Yamijala commented on HADOOP-3759:
------------------------------------------

Ari, I had looked at the ResourceEstimator class that you defined in HADOOP-657. In this patch we are not estimating the used memory - rather using a configured value. If we go to a model where we estimate memory per task, then we should definitely use the ResourceEstimator. But for now, we are going with a simpler approach of letting the user specify this to us in configuration.

On the other hand, to communicate the used disk space, you are following roughly the same mechanism as what is defined here IIRC - which is to compute the free disk space in the heartbeat and communicate it via the TaskTrackerStatus. Vinod's comment was to unify your change as part of the resource map defined here so that it would be done in a like manner.

Makes sense ?

> Provide ability to run memory intensive jobs without affecting other running tasks on the nodes
> -----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3759
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3759
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>             Fix For: 0.19.0
>
>         Attachments: HADOOP-3759.patch, HADOOP-3759.patch, HADOOP-3759.patch
>
>
> In HADOOP-3581, we are discussing how to prevent memory intensive tasks from affecting Hadoop daemons and other tasks running on a node. A related requirement is that users be provided an ability to run jobs which are memory intensive. The system must provide enough knobs to allow such jobs to be run while still maintaining the requirements of HADOOP-3581.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.