hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean (Contractor)" <dean.hil...@broadridge.com>
Subject RE: External jar dependency of map reduce jobs
Date Wed, 29 Dec 2010 23:02:37 GMT
Google DistributedCache hadoop...I think there is a method for
DistributedCache.addResource though not sure if adding just text files
or jars, but maybe that will help you in looking.  Sorry, I am a newb as



From: praveen.peddi@nokia.com [mailto:praveen.peddi@nokia.com] 
Sent: Wednesday, December 29, 2010 8:40 AM
To: mapreduce-user@hadoop.apache.org
Subject: External jar dependency of map reduce jobs


Hello all,

I have few map reduce jobs that I am invoking from a glasfish container.
I am not using "hadoop" command line tool but calling the jobs directly
from glassfish programatically. I have a driver that runs on glassfish
and calls these jobs sequentially. I was able to run the jobs as long as
they do not depend on external jar files. But I have one job that
depends on external jars and that fails with ClassNotFound exception.


I know that -libjars options can be used when using command line hadoop
but since I am invoking the jobs programatically from an app server, I
am wondering how I can add external dependencies in this case.


Any help would be appreciated.





This message and any attachments are intended only for the use of the addressee and
may contain information that is privileged and confidential. If the reader of the 
message is not the intended recipient or an authorized representative of the
intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication in
error, please notify us immediately by e-mail and delete the message and any
attachments from your system.

View raw message