hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vipul Pandey <vipan...@gmail.com>
Subject Re: -libjars?
Date Sat, 11 Dec 2010 23:19:10 GMT
Sweeet!  That solved the problem! I was assuming that there must be a classloader or something
that loads the -libjars locally as well. i was wrong after-all! 
Thank You.
~V
On Dec 10, 2010, at 10:08 AM, Todd Lipcon wrote:

> You need to put the library jar on your classpath (eg using
> HADOOP_CLASSPATH) as well. The -libjars will ship it to the cluster
> and put it on the classpath of your task, but not the classpath of
> your "driver" code.
> 
> -Todd
> 
> On Thu, Dec 9, 2010 at 10:29 PM, Vipul Pandey <vipandey@gmail.com> wrote:
>> disclaimer : a newbie!!!
>> Howdy?
>> Got a quick question. -libjars option doesn't seem to work for me in -
>> prettymuch - my first (or mayby second) mapreduce job.
>> Here's what i'm doing :
>> $bin/hadoop jar  sherlock.jar somepkg.FindSchoolsJob -libjars
>>  HStats-1A18.jar input output
>> 
>> sherlock.jar has my main class (ofcourse)  FindSchoolsJob, which runs just
>> fine by itself till I add a dependency on a class in HStats-1A18.jar.
>> When I run the above command with -libjars specified - it fails to find my
>> classes that 'are' inside HStats jar file.
>> Exception in thread "main" java.lang.NoClassDefFoundError: com/*****/HAgent
>> at com.*****.FindSchoolsJob.run(FindSchoolsJob.java:46)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at com.******.FindSchoolsJob.main(FindSchoolsJob.java:101)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>> Caused by: java.lang.ClassNotFoundException:com/*****/HAgent
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>> ... 8 more
>> 
>> My main class is defined as below :
>> public class FindSchoolsJob extends Configured implements Tool {
>> :
>> public int run(String[] args) throws Exception {
>> :
>> :
>>               }
>> :
>> public static void main(String[] args) throws Exception {
>> int res = ToolRunner.run(new Configuration(), new FindSchoolsJob(),
>> args);
>> System.exit(res);
>> }
>> }
>> Any hint would be highly appreciated.
>> Thank You!
>> ~V
> 
> 
> 
> -- 
> Todd Lipcon
> Software Engineer, Cloudera


Mime
View raw message