Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <71209857.1225734764431.JavaMail.jira@brutus>
Date: Mon, 3 Nov 2008 09:52:44 -0800 (PST)
From: "Doug Cutting (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-4035) Modify the capacity scheduler
 (HADOOP-3445) to schedule tasks based on memory requirements and task
 trackers free memory
In-Reply-To: <1827797122.1219839824462.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644769#action_12644769 ] 

Doug Cutting commented on HADOOP-4035:
--------------------------------------

Allen> I think the system should be smart enough that if I specify "4g" or "4m" or "4k" it knows what I'm talking about.

Steve> There's a lot to be said for configuration languages with basic math operators [ ... ]

These sound like great enhancements to the Configuration API.  The getInt() and getLong() methods could implement these.  But that should probably be in a separate issue.  And, if it's to be implemented, the parameter in this issue should then be raw bytes, no?


> Modify the capacity scheduler (HADOOP-3445) to schedule tasks based on memory requirements and task trackers free memory
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-4035
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4035
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>    Affects Versions: 0.19.0
>            Reporter: Hemanth Yamijala
>            Assignee: Vinod K V
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: 4035.1.patch, HADOOP-4035-20080918.1.txt, HADOOP-4035-20081006.1.txt, HADOOP-4035-20081006.txt, HADOOP-4035-20081008.txt
>
>
> HADOOP-3759 introduced configuration variables that can be used to specify memory requirements for jobs, and also modified the tasktrackers to report their free memory. The capacity scheduler in HADOOP-3445 should schedule tasks based on these parameters. A task that is scheduled on a TT that uses more than the default amount of memory per slot can be viewed as effectively using more than one slot, as it would decrease the amount of free memory on the TT by more than the default amount while it runs. The scheduler should make the used capacity account for this additional usage while enforcing limits, etc.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.