hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Grant Overby (groverby)" <grove...@cisco.com>
Subject Re: issue about enable UseNUMA flag in hadoop framework
Date Mon, 21 Sep 2015 15:18:11 GMT
UseNUMA isn't supported by all garbage collectors.

CMS is not supported
G1 is supported
Not sure about others


Grant Overby
Software Engineer
Mobile: 865 724 4910

[http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif] Think before you print.

This email may contain confidential and privileged material for the sole use of the intended
recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If
you are not the intended recipient (or authorized to receive for the recipient), please contact
the sender by reply email and delete all copies of this message.

Please click here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html>
for Company Registration Information.

From: Zhudacai <zhudacai@hisilicon.com<mailto:zhudacai@hisilicon.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Saturday, September 19, 2015 at 4:42 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>,
"general@hadoop.apache.org<mailto:general@hadoop.apache.org>" <general@hadoop.apache.org<mailto:general@hadoop.apache.org>>,
"common-issues@hadoop.apache.org<mailto:common-issues@hadoop.apache.org>" <common-issues@hadoop.apache.org<mailto:common-issues@hadoop.apache.org>>,
"yarn-issues@hadoop.apache.org<mailto:yarn-issues@hadoop.apache.org>" <yarn-issues@hadoop.apache.org<mailto:yarn-issues@hadoop.apache.org>>,
"mapreduce-issues@hadoop.apache.org<mailto:mapreduce-issues@hadoop.apache.org>." <mapreduce-issues@hadoop.apache.org<mailto:mapreduce-issues@hadoop.apache.org>>
Cc: "dongbo (E)" <dongbo4@huawei.com<mailto:dongbo4@huawei.com>>, Zhanweitao <zhanweitao@hisilicon.com<mailto:zhanweitao@hisilicon.com>>
Subject: issue about enable UseNUMA flag in hadoop framework


We get a problem about enable UseNUMA flag for my hadoop framework.

We've tried to specify JVM flags during hadoop daemon's starts,
e.g. export HADOOP_NAMENODE_OPTS="-XX:+UseNUMA -Dcom.sun.management.jmxremote $HADOOP_NAMENODE_OPTS",
But the ratio between local and remote memory access is 2:1, just remains as same as before.

Then we find that hadoop MapReduce start child JVM processes to run task in containers. So
we passes -XX:+UseNUMA to JVMs by set theting configuration parameter child.java.opts. But
hadoop starts to throw ExitCodeExceptionException (exitCode=1), seems that hadoop does not
support this JVM parameter.

What should we do to enable UseNUMA flag for my hadoop? Or what should we do to decrease the
local/remote memory access in NUMA framework? Should we just change Hadoop script or resorts
to source code? And how to do it?

The hadoop version is 2.6.0.

Best Regards.


View raw message