hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joey Echeverria <j...@cloudera.com>
Subject Re: Hadoop Mapreduce jobs and LD_LIBRARY_PATH
Date Fri, 29 Apr 2011 15:55:54 GMT
Just to confirm, you restarted hadoop after making the changes to
mapred-site.xml?

-Joey

On Fri, Apr 29, 2011 at 11:53 AM, Donatella Firmani
<donatellafirmani@yahoo.com> wrote:
> Hi Alex,
>
> I'm just editing mapred-site.xml in /conf directory of my hadoop
> installation root.
> I'm running in pseudo-distributed mode?
>
> Should I edit something else?
>
> Thanks for your quick reply (I'm getting crazy with this problem :-P)
>
> Regards,
> DF
>
> ________________________________
> From: Alex Kozlov <alexvk@cloudera.com>
> To: mapreduce-user@hadoop.apache.org
> Sent: Fri, April 29, 2011 5:45:55 PM
> Subject: Re: Hadoop Mapreduce jobs and LD_LIBRARY_PATH
>
> Hi Donatella, Are you sure you are passing the parameters the correct way?
> Do you see these set in your job.xml file (on a slave)?
>
> On Fri, Apr 29, 2011 at 8:41 AM, Donatella Firmani
> <donatellafirmani@yahoo.com> wrote:
>>
>> In any case - I tried different sitaxes - I have the same result.
>> I made my mapper process dump on log files the result of
>>
>> System.getenv("LD_LIBRARY_PATH")
>> System.getProperty("java.library.path")
>>
>> and none of the values seem to be affected neither by the setting of
>> mapred.child.java.opts or of mapred.child.env. :-(
>>
>> Maybe hadoop ovverrides something at run time or at start time??
>>
>> Thanks in advance,
>> DF
>>
>>
>> ________________________________
>> From: Donatella Firmani <donatellafirmani@yahoo.com>
>> To: mapreduce-user@hadoop.apache.org
>> Sent: Fri, April 29, 2011 5:17:00 PM
>> Subject: Re: Hadoop Mapreduce jobs and LD_LIBRARY_PATH
>>
>> Dear Robert,
>>
>> thanks for your quick reply. So you are saying that i can add a property
>> in hadoop-0.20.2 adding an item in mapred-site.xml. I have two questions
>> related to the syntax, do you confirm me that
>>
>> 1) different path are to be separated by a ":" like in unix systems or by
>> a ";"?
>> 2) the blank space is link LD_LIBRARY_PATH = /home/mylibpath1/lib1
>>    (and so LD_LIBRARY_PATH=/home/mylibpath1/lib1 is wrong)
>>
>> ?
>>
>> Thanks in advance,
>> DF
>>
>> ________________________________
>> From: Robert Evans <evans@yahoo-inc.com>
>> To: "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org>
>> Sent: Fri, April 29, 2011 4:29:51 PM
>> Subject: Re: Hadoop Mapreduce jobs and LD_LIBRARY_PATH
>>
>> DF,
>>
>> You can set mapred.child.java.opts to set java options, but you can also
>> set mapred.child.env to set environment variables, be careful because they
>> are space separated with an = in between them.
>>
>>      <property>
>>
>>    <name>mapred.child.env</name>
>>
>>     <value>LD_LIBRARY_PATH=/home/mylibpath1/lib1;home/mylibpath2/lib2;home/mylibpath3/lib3</value>
>>
>>     </property>
>>
>> --Bobby
>>
>> On 4/29/11 5:58 AM, "Donatella Firmani" <donatellafirmani@yahoo.com>
>> wrote:
>>
>> To solve the issue addressed in my previous message, i tried setting
>> property
>> mapred.child.java.opts in mapred-site.xml. But - even if it seems the
>> right
>> approach in relation to what said in blogs & forums - there is a big
>> problem
>> with it.
>>
>> Following the tutorial (hadoop website) as section Task Execution &
>> Environment,
>>
>> my xml looks like:
>>
>> <configuration>
>>      <property>
>>          <name>mapred.job.tracker</name>
>>          <value>localhost:9001</value>
>>      </property>
>>      <property>
>>          <name>mapred.child.java.opts</name>
>>          <value>
>>
>> -Djava.library.path=/home/mylibpath1/lib1;home/mylibpath2/lib2;home/mylibpath3/lib3
>>
>>
>>          </value>
>>      </property>
>> </configuration>
>>
>> The problem arises when executing the job, because it is thrown an
>> exception:
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>
>> -Djava/library/path=/home/mylibpath1/lib1;home/mylibpath2/lib2;home/mylibpath3/lib3
>>
>>
>>
>> Any help would be appreciated.
>> Thanks in advance,
>>
>> DF
>>
>>
>>
>> ----- Original Message ----
>> From: Donatella Firmani <donatellafirmani@yahoo.com>
>> To: mapreduce-user@hadoop.apache.org
>> Sent: Fri, April 29, 2011 12:57:52 PM
>> Subject: Hadoop Mapreduce jobs and LD_LIBRARY_PATH
>>
>>
>>
>> Hi to all,
>>
>> I just subscribed to this mailing list and I'd like to ask you if anyone
>> knows
>> how to deal with LD_LIBRARY_PATH.
>> I have a Java application that needs a proper setting of this environment
>> variable to work under Linux-Ubuntu.
>> I want to use tis application from a mapreduce job, unfortunately I could
>> not
>> find a way to make things work against the LD_LIBRARY_PATH environment
>> variable.
>>
>> I tried so many different strategies and I am stuck. Maybe someone of you
>> can
>> help.
>>
>> Thanks in advance,
>> Cheers.
>>
>> DF
>>
>> PS: I use hadoop-0-20-2
>>
>
>



-- 
Joseph Echeverria
Cloudera, Inc.
443.305.9434

Mime
View raw message