hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepika Khera <Deepika.Kh...@avg.com>
Subject RE: System property in hadoop
Date Mon, 03 May 2010 20:34:48 GMT
Thanks Alex! Yes, that worked.... I had tried using the "mapred.child.java.opts" property to
pass the variable earlier, but missed out something else due to which it did not work...Now
that you mentioned, I verified it and found out my mistake.

Thanks again,

From: Alex Kozlov [mailto:alexvk@cloudera.com]
Sent: Friday, April 30, 2010 6:23 PM
To: mapreduce-user@hadoop.apache.org
Subject: Re: System property in hadoop

Hi Deepika,

If you want to pass a java system property to a task running on a tasktracker, you should
set mapred.child.java.opts in a Hadoop Configuration<http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/conf/Configuration.html>
object like:

$ hadoop jar <your-jar> -D mapred.child.java.opts="-Dmy.file.name<http://dmy.file.name/>=/abc/def.properties"

Your main class has to call ToolRunner.run()<http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/util/ToolRunner.html>.

Let me know if this helps and have a nice weekend.

Alex K
On Fri, Apr 30, 2010 at 3:46 PM, Deepika Khera <Deepika.Khera@avg.com<mailto:Deepika.Khera@avg.com>>

I want to add a Java System property that a running reduce task should be able to read . I
added it to the hadoop-env.sh script in the HADOOP_OPTS.

So, like

export HADOOP_OPTS="-Dmy.file.name<http://Dmy.file.name>=/abc/def.properties"

But still the task is not able to read the property (System.getProperty("my.file.name<http://my.file.name>")
in the reduce task fails ) . Am I missing something, is this not the right place to specify
a java system property ?


View raw message