hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Eric CAZAMEA <jecaza...@yahoo.com>
Subject Re: issue about enable UseNUMA flag in hadoop framework
Date Sun, 20 Sep 2015 11:14:26 GMT
unsubscribe 


     On Saturday, September 19, 2015 10:42 AM, Zhudacai <zhudacai@hisilicon.com> wrote:
   

  <!--#yiv7661362184 _filtered #yiv7661362184 {panose-1:2 1 6 0 3 1 1 1 1 1;} _filtered
#yiv7661362184 {font-family:"Cambria Math";panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv7661362184
{font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv7661362184 {panose-1:2
1 6 0 3 1 1 1 1 1;}#yiv7661362184 #yiv7661362184 p.yiv7661362184MsoNormal, #yiv7661362184
li.yiv7661362184MsoNormal, #yiv7661362184 div.yiv7661362184MsoNormal {margin:0cm;margin-bottom:.0001pt;text-align:justify;text-justify:inter-ideograph;font-size:10.5pt;font-family:"Calibri",
"sans-serif";}#yiv7661362184 a:link, #yiv7661362184 span.yiv7661362184MsoHyperlink {color:blue;text-decoration:underline;}#yiv7661362184
a:visited, #yiv7661362184 span.yiv7661362184MsoHyperlinkFollowed {color:purple;text-decoration:underline;}#yiv7661362184
p.yiv7661362184MsoPlainText, #yiv7661362184 li.yiv7661362184MsoPlainText, #yiv7661362184 div.yiv7661362184MsoPlainText
{margin:0cm;margin-bottom:.0001pt;font-size:10.5pt;font-family:"Calibri", "sans-serif";}#yiv7661362184
span.yiv7661362184EmailStyle17 {font-family:"Calibri", "sans-serif";color:windowtext;}#yiv7661362184
span.yiv7661362184Char {font-family:"Calibri", "sans-serif";}#yiv7661362184 .yiv7661362184MsoChpDefault
{}#yiv7661362184 _filtered #yiv7661362184 {margin:72.0pt 90.0pt 72.0pt 90.0pt;}#yiv7661362184
div.yiv7661362184WordSection1 {}-->Hi,ALL     We get a problem about enable UseNUMA flag
for my hadoop framework.    We've tried to specify JVM flags during hadoop daemon's starts,
e.g. export HADOOP_NAMENODE_OPTS="-XX:+UseNUMA -Dcom.sun.management.jmxremote $HADOOP_NAMENODE_OPTS",
export HADOOP_SECONDARYNAMENODE_OPTS="-XX:+UseNUMA -Dcom.sun.management.jmxremote $HADOOP_SECONDARYNAMENODE_OPTS",
etc. But the ratio between local and remote memory access is 2:1, just remains as same as
before.    Then we find that hadoop MapReduce start child JVM processes to run task in containers.
So we passes -XX:+UseNUMA to JVMs by set theting configuration parameter child.java.opts.
But hadoop starts to throw ExitCodeExceptionException (exitCode=1), seems that hadoop does
not support this JVM parameter.    What should we do to enable UseNUMA flag for my hadoop?
Or what should we do to decrease the local/remote memory access in NUMA framework? Should
we just change Hadoop script or resorts to source code? And how to do it?    The hadoop version
is 2.6.0.    Best Regards.    Dacai 

  
Mime
View raw message