Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Fri, 2 Mar 2012 22:20:00 +0000 (UTC)
From: "Thomas Graves (Commented) (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <1849993124.14597.1330726800165.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1855602063.6359.1329946908680.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (MAPREDUCE-3897) capacity scheduler -
 maxActiveApplicationsPerUser calculation can be wrong
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221303#comment-13221303 ] 

Thomas Graves commented on MAPREDUCE-3897:
------------------------------------------

I should add, my opinion is the other issue with user minimum limit % can be fixed in a separate jira.
                
> capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong
> --------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3897
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3897
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Thomas Graves
>            Assignee: Eric Payne
>            Priority: Critical
>         Attachments: MAPREDUCE-3897-1.txt, MAPREDUCE-3897-1.txt, MAPREDUCE-3897-2.txt
>
>
> The capacity scheduler calculates the maxActiveApplications and the maxActiveApplicationsPerUser based on the config yarn.scheduler.capacity.maximum-applications or default 10000.  
> MaxActiveApplications = max ( ceil ( clusterMemory/minAllocation * maxAMResource% * absoluteMaxCapacity), 1)  
> MaxActiveAppsPerUser = max( ceil (maxActiveApplicationsComputedAbove * (userLimit%/100) * userLimitFactor), 1) 
> maxActiveApplications is already multiplied by the queue absolute MAXIMUM capacity, so if max capacity > capacity and if you have user limit factor 1 (which is the default) and only 1 user is running, that user will not be allowed to use over the queue capacity, so having it relative to MAX capacity doesn't make sense.  That user could easily end up in a deadlock and all its space used by application masters.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira