hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3946) If a resource requirement is higher than available on any node, job should fail early
Date Thu, 01 Mar 2012 06:38:06 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219842#comment-13219842
] 

Todd Lipcon commented on MAPREDUCE-3946:
----------------------------------------

Can we automatically bound the max-resource-size configuration by max(available resources)
across the registered NMs? It seems the most common reason one would hit this is if a newbie
admin set the NM resource limit to something like 1G. In that case, the admin wouldn't know
to also set any max-resource-size parameter.

Or perhaps we can default the max-resource-size parameter to be equal to the advertised NM
resource size, for the same effect?
                
> If a resource requirement is higher than available on any node, job should fail early
> -------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3946
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3946
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 0.24.0, 0.23.2
>            Reporter: Todd Lipcon
>
> If you configure the NMs to have 1GB of RAM each, and then try to submit a job which
has an AM resource requirement of 1.5GB, the job will neither run nor fail. Instead, it will
slowly sop of all of the resources in the cluster as "reservations" despite the fact that
it will never be able to schedule something. Instead, it should fail early indicating that
the required memory allocation is infeasible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message