hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Saptarshi Guha" <saptarshi.g...@gmail.com>
Subject Re: Problem loading hadoop-site.xml - dumping parameters
Date Mon, 05 Jan 2009 19:26:37 GMT
Hello,
I have set my HADOOP_CONF_DIR to the conf folder and still not
loading. I have to manually set the options when I create my conf.
Have you resolved this?

Regards
Saptarshi

On Tue, Dec 30, 2008 at 5:25 PM, g00dn3ss <g00dn3ss@gmail.com> wrote:
> Hey all,
>
> I have a similar issue.  I am specifically having problems with the config
> option "mapred.child.java.opts."  I set it to -Xmx1024m and it uses -Xmx200m
> regardless.  I am running Hadoop 0.18.2 and I'm pretty sure this option was
> working in the previous versions of Hadoop I was using.
>
> I am not explicitly setting HADOOP_CONF_DIR.  My site config is in
> ${HADOOP_HOME}/conf.  Just to test things further, I wrote a small map task
> to print out the ENV values and it has the correct value for HADOOP_HOME,
> HADOOP_LOG_DIR, HADOOP_OPTS, etc...  I also printed out the key/values in
> the JobConf passed to the mapper and it has my specified values for
> fs.default.name and mapred.job.tracker.  Other settings like dfs.name.dir,
> dfs.data.dir, and mapred.child.java.opts do not have my values.
>
> Any suggestion where to look at next?
>
> Thanks!
>
>
>
> On Mon, Dec 29, 2008 at 10:27 PM, Amareshwari Sriramadasu <
> amarsri@yahoo-inc.com> wrote:
>
>> Saptarshi Guha wrote:
>>
>>> Hello,
>>> I had previously emailed regarding heap size issue and have discovered
>>> that the hadoop-site.xml is not loading completely, i.e
>>>  Configuration defaults = new Configuration();
>>>        JobConf jobConf = new JobConf(defaults, XYZ.class);
>>>        System.out.println("1:"+jobConf.get("mapred.child.java.opts"));
>>>        System.out.println("2:"+jobConf.get("mapred.map.tasks"));
>>>        System.out.println("3:"+jobConf.get("mapred.reduce.tasks"));
>>>
>>>  System.out.println("3:"+jobConf.get("mapred.tasktracker.reduce.tasks.maximum"));
>>>
>>> returns -Xmx200m, 2,1,2 respectively, even though the numbers in the
>>> hadoop-site.xml are very different.
>>>
>>> Is there a way for hadoop to dump the parameters read in from
>>> hadoop-site.xml and hadoop-default.xml?
>>>
>>>
>>>
>> Is your hadoop-site.xml present in the conf (HADOOP_CONF_DIR) directory?
>> http://hadoop.apache.org/core/docs/r0.19.0/cluster_setup.html#Configuration
>>
>> -Amareshwari
>>
>



-- 
Saptarshi Guha - saptarshi.guha@gmail.com

Mime
View raw message