hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-566) Add a misconfiguration diagnostic or highlight mismatched resource.mb vs -Xmx
Date Thu, 11 Apr 2013 10:07:16 GMT
Gopal V created YARN-566:
----------------------------

             Summary: Add a misconfiguration diagnostic or highlight mismatched resource.mb
vs -Xmx
                 Key: YARN-566
                 URL: https://issues.apache.org/jira/browse/YARN-566
             Project: Hadoop YARN
          Issue Type: Improvement
            Reporter: Gopal V
            Priority: Minor


It is possible to misconfigure a pure-java MR application by setting mapreduce resource size
to 1536Mb, while leaving the -Xmx value at the default 512Mb.

The AM UI can track over-all memory usage of an app either via Runtime.maxMemory() or via
the process watcher in YARN to report the actual memory used by the application.

MR does not fail even when the memory -Xmx is set lower than the YARN  allocations, but it
spills & GCs instead to keep it with in the memory sizes.

This is sub-optimal and would make sense for YARN to highlight in the AM UI or JHS UI for
tuning/debugging purposes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message