hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Kozlov <ale...@cloudera.com>
Subject Re: Hadoop Mapreduce jobs and LD_LIBRARY_PATH
Date Fri, 29 Apr 2011 15:45:55 GMT
Hi Donatella, Are you sure you are passing the parameters the correct way?
Do you see these set in your job.xml file (on a slave)?

On Fri, Apr 29, 2011 at 8:41 AM, Donatella Firmani <
donatellafirmani@yahoo.com> wrote:

> In any case - I tried different sitaxes - I have the same result.
> I made my mapper process dump on log files the result of
>
> System.getenv("LD_LIBRARY_PATH")
> System.getProperty("java.library.path")
>
> and none of the values seem to be affected neither by the setting of mapred.child.java.opts
> or of mapred.child.env. :-(
>
> Maybe hadoop ovverrides something at run time or at start time??
>
> Thanks in advance,
> DF
>
>
> ------------------------------
> *From:* Donatella Firmani <donatellafirmani@yahoo.com>
> *To:* mapreduce-user@hadoop.apache.org
> *Sent:* Fri, April 29, 2011 5:17:00 PM
>
> *Subject:* Re: Hadoop Mapreduce jobs and LD_LIBRARY_PATH
>
> Dear Robert,
>
> thanks for your quick reply. So you are saying that i can add a property in
> hadoop-0.20.2 adding an item in mapred-site.xml. I have two questions
> related to the syntax, do you confirm me that
>
> 1) different path are to be separated by a ":" like in unix systems or by a
> ";"?
> 2) the blank space is link LD_LIBRARY_PATH = /home/mylibpath1/lib1
>    (and so LD_LIBRARY_PATH=/home/mylibpath1/lib1 is wrong)
>
> ?
>
> Thanks in advance,
> DF
>
> ------------------------------
> *From:* Robert Evans <evans@yahoo-inc.com>
> *To:* "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org
> >
> *Sent:* Fri, April 29, 2011 4:29:51 PM
> *Subject:* Re: Hadoop Mapreduce jobs and LD_LIBRARY_PATH
>
> DF,
>
> You can set mapred.child.java.opts to set java options, but you can also
> set mapred.child.env to set environment variables, be careful because they
> are space separated with an = in between them.
>
>      <property>
>
>    <name>mapred.child.env</name>
>
>     <value>LD_LIBRARY_PATH=/home/mylibpath1/lib1;home/mylibpath2/lib2;home/mylibpath3/lib3</value>
>
>     </property>
>
> --Bobby
>
> On 4/29/11 5:58 AM, "Donatella Firmani" <donatellafirmani@yahoo.com>
> wrote:
>
> To solve the issue addressed in my previous message, i tried setting
> property
> mapred.child.java.opts in mapred-site.xml. But - even if it seems the right
> approach in relation to what said in blogs & forums - there is a big
> problem
> with it.
>
> Following the tutorial (hadoop website) as section Task Execution &
> Environment,
>
> my xml looks like:
>
> <configuration>
>      <property>
>          <name>mapred.job.tracker</name>
>          <value>localhost:9001</value>
>      </property>
>      <property>
>          <name>mapred.child.java.opts</name>
>          <value>
>
> -Djava.library.path=/home/mylibpath1/lib1;home/mylibpath2/lib2;home/mylibpath3/lib3
>
>
>          </value>
>      </property>
> </configuration>
>
> The problem arises when executing the job, because it is thrown an
> exception:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
>
> -Djava/library/path=/home/mylibpath1/lib1;home/mylibpath2/lib2;home/mylibpath3/lib3
>
>
>
> Any help would be appreciated.
> Thanks in advance,
>
> DF
>
>
>
> ----- Original Message ----
> From: Donatella Firmani <donatellafirmani@yahoo.com>
> To: mapreduce-user@hadoop.apache.org
> Sent: Fri, April 29, 2011 12:57:52 PM
> Subject: Hadoop Mapreduce jobs and LD_LIBRARY_PATH
>
>
>
> Hi to all,
>
> I just subscribed to this mailing list and I'd like to ask you if anyone
> knows
> how to deal with LD_LIBRARY_PATH.
> I have a Java application that needs a proper setting of this environment
> variable to work under Linux-Ubuntu.
> I want to use tis application from a mapreduce job, unfortunately I could
> not
> find a way to make things work against the LD_LIBRARY_PATH environment
> variable.
>
> I tried so many different strategies and I am stuck. Maybe someone of you
> can
> help.
>
> Thanks in advance,
> Cheers.
>
> DF
>
> PS: I use hadoop-0-20-2
>
>

Mime
View raw message