hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mahadev konar (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3483) CapacityScheduler reserves container on same node as AM but can't ever use due to never enough avail memory
Date Wed, 30 Nov 2011 19:47:40 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160276#comment-13160276
] 

Mahadev konar commented on MAPREDUCE-3483:
------------------------------------------

MAPREDUCE-2917 actually tried to solve this exact issue. 
                
> CapacityScheduler reserves container on same node as AM but can't ever use due to never
enough avail memory
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3483
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3483
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Thomas Graves
>            Priority: Blocker
>
> Saw a case where a job was stuck trying to get reducers.  The issue is the capacity scheduler
reserved a container on the same node as the application master but there wasn't ever enough
memory to run the reducer on that node.  Node total memory was 8G, Reducer needed 8G, AM was
using 2G.  This particular job had 10 reducers and it was stuck waiting on the one because
the AM + reserved reducer memory was already over the queue limit.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message