hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Praveen Sripati <praveensrip...@gmail.com>
Subject Re: External libraries usage
Date Wed, 28 Dec 2011 11:35:34 GMT
Check this article from Cloudera on different ways of distributing a jar
file to the job.

http://www.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/

Praveen

On Wed, Dec 28, 2011 at 5:40 AM, Eyal Golan <egolan74@gmail.com> wrote:

> Hello,
> Another newbie question.
> Suppose I want to use an external library (jar) in the mapper / reducer
> classes.
> (commons-lang, google's guava, etc.)
> In our environment, I added the jars into a specific folder and added them
> to HADOOP-CLASSPATH.
> However, when running mapper that uses one of the jars, it could not find
> the classes in that jar.
>
> I thought that it might be in our environment (I am not managing our
> cluster).
>
> Then I read about DistributedCache.
> Should I use it with methods such as addArchiveToClassPath,
> addFileToClassPath, addCachArchive to use jar libraries?
> If so, which method is more appropriate ?
>
> If not, how do we load jar libraries to each VM?
>
> Thanks,
>
> Eyal Golan
> egolan74@gmail.com
>
> Visit: http://jvdrums.sourceforge.net/
> LinkedIn: http://www.linkedin.com/in/egolan74
> Skype: egolan74
>
> P  Save a tree. Please don't print this e-mail unless it's really necessary
>
>

Mime
View raw message