hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "刘晓文" <lxw1...@qq.com>
Subject Re: Job driver and 3rd party jars
Date Fri, 08 Mar 2013 07:15:30 GMT
try:
hadoop jar .... -Dmapreduce.task.classpath.user.precedence=true -libjars your_jar 





------------------ Original ------------------
From:  "Barak Yaish"<barak.yaish@gmail.com>;
Date:  Fri, Mar 8, 2013 03:06 PM
To:  "user"<user@hadoop.apache.org>; 
Subject:  Re: Job driver and 3rd party jars




Yep, my typo, I'm using the later. I was also trying export HADOOP_CLASSPATH_USER_FIRST =true
and export HADOOP_CLASSPATH=myjar before launching the hadoop jar, but I still getting the
same exception.
 I'm running hadoop 1.0.4.
 On Mar 8, 2013 2:27 AM, "Harsh J" <harsh@cloudera.com> wrote:
  To be precise, did you use -libjar or -libjars? The latter is the right option.
 
 On Fri, Mar 8, 2013 at 12:18 AM, Barak Yaish <barak.yaish@gmail.com> wrote:
 > Hi,
 >
 > I'm able to run M/R jobs where the mapper and reducer required to use 3rd
 > party jars. I'm registering those jars in -libjar while invoking the hadoop
 > jar command. I'm facing a strange problem, though, when the job driver
 > itself ( extends Configured implements Tool ) required to run such code (
 > for example notify some remote service upon start and end). Is there a way
 > to configure classpath when submitting jobs using hadoop jar? Seems like
 > -libjar doesn't work for this case...
 >
 > Exception in thread "main" java.lang.NoClassDefFoundError:
 > com/me/context/DefaultContext
 >         at java.lang.ClassLoader.defineClass1(Native Method)
 >         at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
 >         at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
 >         at
 > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
 >         at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
 >         at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
 >         at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
 >         at java.security.AccessController.doPrivileged(Native Method)
 >         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 >         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
 >         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
 >         at
 > com.peer39.bigdata.mr.pnm.PnmDataCruncher.run(PnmDataCruncher.java:50)
 >         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
 >         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
 >         at com.me.mr.pnm.PnmMR.main(PnmDataCruncher.java:261)
 >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 >         at
 > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 >         at
 > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 >         at java.lang.reflect.Method.invoke(Method.java:597)
 >         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
 > Caused by: java.lang.ClassNotFoundException: com.me.context.DefaultContext
 >         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 >         at java.security.AccessController.doPrivileged(Native Method)
 >         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 >         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
 >         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
 
 
 
 --
 Harsh J
Mime
View raw message