hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinay (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9870) Mixed configurations for JVM -Xmx in hadoop command
Date Wed, 20 Nov 2013 05:37:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13827363#comment-13827363
] 

Vinay commented on HADOOP-9870:
-------------------------------

bq. I haven't found any documents that said the jvm would pick the last one
Yes you are right. I too dint find any explicit document in hadoop mentioned about that. But
we tested it and found that later argument value only it will use. And we are using in our
clusters by configuring higher value than default of 1000m. 
User specied opts are added at last of the command line list just before the classname just
make sure that their parameters take effect

> Mixed configurations for JVM -Xmx in hadoop command
> ---------------------------------------------------
>
>                 Key: HADOOP-9870
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9870
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Wei Yan
>         Attachments: HADOOP-9870.patch, HADOOP-9870.patch, HADOOP-9870.patch
>
>
> When we use hadoop command to launch a class, there are two places setting the -Xmx configuration.
> *1*. The first place is located in file {{hadoop-common-project/hadoop-common/src/main/bin/hadoop}}.
> {code}
> exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"
> {code}
> Here $JAVA_HEAP_MAX is configured in hadoop-config.sh ({{hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh}}).
The default value is "-Xmx1000m".
> *2*. The second place is set with $HADOOP_OPTS in file {{hadoop-common-project/hadoop-common/src/main/bin/hadoop}}.
> {code}
> HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS"
> {code}
> Here $HADOOP_CLIENT_OPTS is set in hadoop-env.sh ({{hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh}})
> {code}
> export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS"
> {code}
> Currently the final default java command looks like:
> {code}java -Xmx1000m .... -Xmx512m CLASS_NAME ARGUMENTS"{code}
> And if users also specify the -Xmx in the $HADOOP_CLIENT_OPTS, there will be three -Xmx
configurations. 
> The hadoop setup tutorial only discusses hadoop-env.sh, and it looks that users should
not make any change in hadoop-config.sh.
> We should let hadoop smart to choose the right one before launching the java command,
instead of leaving for jvm to make the decision.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message