hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur" <tuc...@gmail.com>
Subject Re: Working with external libraries?
Date Sun, 24 Feb 2008 11:10:52 GMT
If you add the native libraries to the distributed cache on the job
working directory they will be picked up for the job. no need to
modify anything in hadoop.

This functionality was introduced with in  HADOOP-1660
(https://issues.apache.org/jira/browse/HADOOP-1660).

You need to add the lib to the distributed cache for the job and then
created a symlink on it.

A


On Sun, Feb 24, 2008 at 1:24 PM, Jason Venner <jason@attributor.com> wrote:
> For our JNI tasks, we just install our libraries in the same places on
>  each machine, and set the LD_LIBRARY_PATH to include these directories,
>  in hadoop-env.sh.
>
>  This does require a restart of the mapred portion of the cluster if you
>  need to change the LD_LIBRARY_PATH.
>
>  Chang Hu wrote:
>  > I believe both static and dynamic libraries can be make via gcc.  Is there a
>  > difference and where should I put them in Hadoop?
>  >
>  > Thanks,
>  >
>  >  - Chang
>  >
>  > On Sat, Feb 23, 2008 at 10:28 AM, 11 Nov. <nov.eleventh@gmail.com> wrote:
>  >
>  >
>  >> Are your external C++ libraries statically linked?
>  >>
>  >> 2008/2/22, Chang Hu <changhu@cs.umd.edu>:
>
> >>
>  >>> Hi,
>  >>>
>  >>> I have an image processing library in C++ and want to run it as a
>  >>> MapReduce
>  >>> job via JNI. While I have some idea about how to include an external JAR
>  >>> into MapReduce, I am not sure how that works with external C++
>  >>>
>  >> libraries.
>  >>
>  >>> It could be easier to use HadoopStreaming, but I am not sure how to do
>  >>> that,
>  >>> either.
>  >>>
>  >>> Suggestions?
>  >>>
>  >>> Thanks,
>  >>>
>  >>>   - Chang
>  >>>
>  >>> --
>  >>> ---------------
>  >>> √úberstehen ist alles.
>  >>>
>  >>>
>  >>>
>  >>> Chang Hu
>  >>> Ph.D. student
>  >>> Computer Science Department
>  >>> University of Maryland
>  >>>
>  >>>
>  >
>  >
>  >
>  >
>
>  --
>  Jason Venner
>  Attributor - Publish with Confidence <http://www.attributor.com/>
>  Attributor is hiring Hadoop Wranglers, contact if interested
>

Mime
View raw message