hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ramya Sunil (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3784) maxActiveApplications(|PerUser) per queue is too low for small clusters
Date Fri, 03 Feb 2012 00:30:53 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13199407#comment-13199407

Ramya Sunil commented on MAPREDUCE-3784:

+1. Looks good. Tested the patch with various combination of maxCap, userlimit and queue capacity.

The problem of apps queuing up for small queues is alleviated after applying the patch. Also
verified that the maxActiveApplications/maxActiveApplicationsPerUser is computed dynamically
based on cluster resources.
> maxActiveApplications(|PerUser) per queue is too low for small clusters
> -----------------------------------------------------------------------
>                 Key: MAPREDUCE-3784
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3784
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: Arun C Murthy
>             Fix For: 0.23.1
>         Attachments: MAPREDUCE-3784.patch, MAPREDUCE-3784.patch, MAPREDUCE-3784.patch,
MAPREDUCE-3784.patch, MAPREDUCE-3784.patch
> We ran into this issue while testing on small clusters. 
> On a 7node cluster with 8G per node,  for a queue with absolute capacity 30%, user limit
100%, maxActiveApplications and maxActiveApplicationsPerUser is calculated to be 1.
> This means that even though the queue has 17GB(0.3*8*7), only 1 user can run 1 app at
a given time queuing up rest of the apps/users. This hurts performance on small clusters.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message