Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of g00dn3ss@gmail.com designates
 74.125.46.29 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:references;
        b=HpTsgGvMa4J20dGlztq5khkYN/qzc+Nfdod8FsIvWxJ7BN7rF0eIdXdsa7cZBJTiFF
         y9IEGDs0yzfjcz4BlRm8bgBveK4qpkylUKsD56Wdt3WqttrjcsU2qVg4pcHSPanrUfnf
         U0+CkB7edlBHi5wKbvrBhGNk3vqXNV2C4xy2M=
Message-ID: <3dc1e8330812301425i1607d51bjed72991ac21018ce@mail.gmail.com>
Date: Tue, 30 Dec 2008 17:25:18 -0500
From: g00dn3ss <g00dn3ss@gmail.com>
To: core-user@hadoop.apache.org
Subject: Re: Problem loading hadoop-site.xml - dumping parameters
In-Reply-To: <49599519.9030402@yahoo-inc.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_118494_11400590.1230675918920"
References: <1e7471d50812291318u4b9e8406ha6875b08b1a150e4@mail.gmail.com>
	 <49599519.9030402@yahoo-inc.com>

------=_Part_118494_11400590.1230675918920
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hey all,

I have a similar issue.  I am specifically having problems with the config
option "mapred.child.java.opts."  I set it to -Xmx1024m and it uses -Xmx200m
regardless.  I am running Hadoop 0.18.2 and I'm pretty sure this option was
working in the previous versions of Hadoop I was using.

I am not explicitly setting HADOOP_CONF_DIR.  My site config is in
${HADOOP_HOME}/conf.  Just to test things further, I wrote a small map task
to print out the ENV values and it has the correct value for HADOOP_HOME,
HADOOP_LOG_DIR, HADOOP_OPTS, etc...  I also printed out the key/values in
the JobConf passed to the mapper and it has my specified values for
fs.default.name and mapred.job.tracker.  Other settings like dfs.name.dir,
dfs.data.dir, and mapred.child.java.opts do not have my values.

Any suggestion where to look at next?

Thanks!


On Mon, Dec 29, 2008 at 10:27 PM, Amareshwari Sriramadasu <
amarsri@yahoo-inc.com> wrote:

> Saptarshi Guha wrote:
>
>> Hello,
>> I had previously emailed regarding heap size issue and have discovered
>> that the hadoop-site.xml is not loading completely, i.e
>>  Configuration defaults = new Configuration();
>>        JobConf jobConf = new JobConf(defaults, XYZ.class);
>>        System.out.println("1:"+jobConf.get("mapred.child.java.opts"));
>>        System.out.println("2:"+jobConf.get("mapred.map.tasks"));
>>        System.out.println("3:"+jobConf.get("mapred.reduce.tasks"));
>>
>>  System.out.println("3:"+jobConf.get("mapred.tasktracker.reduce.tasks.maximum"));
>>
>> returns -Xmx200m, 2,1,2 respectively, even though the numbers in the
>> hadoop-site.xml are very different.
>>
>> Is there a way for hadoop to dump the parameters read in from
>> hadoop-site.xml and hadoop-default.xml?
>>
>>
>>
> Is your hadoop-site.xml present in the conf (HADOOP_CONF_DIR) directory?
> http://hadoop.apache.org/core/docs/r0.19.0/cluster_setup.html#Configuration
>
> -Amareshwari
>

------=_Part_118494_11400590.1230675918920--