hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Kozlov <ale...@cloudera.com>
Subject Re: -libjars?
Date Sat, 11 Dec 2010 23:04:28 GMT
Try using "-files HStats-1A18.jar" as well: it will put it into Distributed
Cache on the HDFS cluster...  -- Alex K

On Fri, Dec 10, 2010 at 10:08 AM, Todd Lipcon <todd@cloudera.com> wrote:

> You need to put the library jar on your classpath (eg using
> HADOOP_CLASSPATH) as well. The -libjars will ship it to the cluster
> and put it on the classpath of your task, but not the classpath of
> your "driver" code.
>
> -Todd
>
> On Thu, Dec 9, 2010 at 10:29 PM, Vipul Pandey <vipandey@gmail.com> wrote:
> > disclaimer : a newbie!!!
> > Howdy?
> > Got a quick question. -libjars option doesn't seem to work for me in -
> > prettymuch - my first (or mayby second) mapreduce job.
> > Here's what i'm doing :
> > $bin/hadoop jar  sherlock.jar somepkg.FindSchoolsJob -libjars
> >  HStats-1A18.jar input output
> >
> > sherlock.jar has my main class (ofcourse)  FindSchoolsJob, which runs
> just
> > fine by itself till I add a dependency on a class in HStats-1A18.jar.
> > When I run the above command with -libjars specified - it fails to find
> my
> > classes that 'are' inside HStats jar file.
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> com/*****/HAgent
> > at com.*****.FindSchoolsJob.run(FindSchoolsJob.java:46)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > at com.******.FindSchoolsJob.main(FindSchoolsJob.java:101)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > Caused by: java.lang.ClassNotFoundException:com/*****/HAgent
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> > ... 8 more
> >
> > My main class is defined as below :
> > public class FindSchoolsJob extends Configured implements Tool {
> > :
> > public int run(String[] args) throws Exception {
> > :
> > :
> >               }
> > :
> > public static void main(String[] args) throws Exception {
> > int res = ToolRunner.run(new Configuration(), new FindSchoolsJob(),
> > args);
> > System.exit(res);
> > }
> > }
> > Any hint would be highly appreciated.
> > Thank You!
> > ~V
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Mime
View raw message