hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Utkarsh Agarwal <unrealutka...@gmail.com>
Subject Re: JNI in MAp REuce
Date Fri, 19 Feb 2010 02:49:28 GMT
My .so file has other .so dependencies , so would I have to add them all in
the DistributedCache . Also I tried setting LD_LIBRARY_PATH in
mapred-site.xml as

<property>
  <name>mapred.child.env</name>
  <value>LD_LIBRARY_PATH=/opt/libs/</value>
 </property>


doesnt work. the java.library.path is not sufficient to set , have to get
LD_LIB set.

-Utkarsh

On Thu, Feb 18, 2010 at 3:14 PM, Allen Wittenauer
<awittenauer@linkedin.com>wrote:

>
>
> Like this:
>
>
> http://hadoop.apache.org/common/docs/current/native_libraries.html#Loading+n
> ative+libraries+through+DistributedCache
>
>
>
> On 2/16/10 5:29 PM, "Jason Rutherglen" <jason.rutherglen@gmail.com> wrote:
>
> > How would this work?
> >
> > On Fri, Feb 12, 2010 at 10:45 AM, Allen Wittenauer
> > <awittenauer@linkedin.com> wrote:
> >>
> >> ... or just use distributed cache.
> >>
> >>
> >> On 2/12/10 10:02 AM, "Alex Kozlov" <alexvk@cloudera.com> wrote:
> >>
> >>> All native libraries should be on each of the cluster nodes.  You need
> to
> >>> set "java.library.path" property to point to your libraries (or just
> put
> >>> them in the default system dirs).
> >>>
> >>> On Fri, Feb 12, 2010 at 9:12 AM, Utkarsh Agarwal
> >>> <unrealutkarsh@gmail.com>wrote:
> >>>
> >>>> Can anybody point me how to use JNI calls in a map reduce program. My
> .so
> >>>> files have other dependencies also , is there a way to load the
> >>>> LD_LIBRARY_PATH for child processes . Should all the native stuff be
> in
> >>>> HDFS?
> >>>>
> >>>> Thanks,
> >>>> Utkarsh.
> >>>>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message