hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harish Mallipeddi <harish.mallipe...@gmail.com>
Subject Re: Help - ClassNotFoundException from tasktrackers
Date Thu, 11 Jun 2009 08:52:16 GMT
Yep that was it. I'd a whitespace after the comma. Thanks Aaron!

On Wed, Jun 10, 2009 at 9:39 PM, Aaron Kimball <aaron@cloudera.com> wrote:

> mapred.local.dir can definitely work with multiple paths, but it's a bit
> strict about the format. You should have one or more paths, separated by
> commas and no whitespace. e.g. "/disk1/mapred,/disk2/mapred". If you've got
> them on new lines, etc, then it might try to interpret that as part of one
> of the paths. Could this be your issue? If you paste your hadoop-site.xml
> in
> to an email we could take a look.
>
> - Aaron
>
> On Wed, Jun 10, 2009 at 2:41 AM, Harish Mallipeddi <
> harish.mallipeddi@gmail.com> wrote:
>
> > Hi,
> >
> > After some amount of debugging, I narrowed down the problem. I'd set
> > multiple paths (multiple disks mounted at different endpoints) for the *
> > mapred.local.dir* setting. When I changed this to just one path,
> everything
> > started working - previously the job jar and the job xml weren't being
> > created under the local/taskTracker/jobCache folder.
> >
> > Any idea why? Am I doing something wrong? I've read somewhere else that
> > specifying multiple disks for mapred.local.dir is important to increase
> > disk
> > bandwidth so that the map outputs get written faster to the local disks
> on
> > the tasktracker nodes.
> >
> > Cheers,
> > Harish
> >
> > On Tue, Jun 9, 2009 at 5:21 PM, Harish Mallipeddi <
> > harish.mallipeddi@gmail.com> wrote:
> >
> > > I setup a new cluster (1 namenode + 2 datanodes). I'm trying to run the
> > > GenericMRLoadGenerator program from hadoop-0.20.0-test.jar. But I keep
> > > getting the ClassNotFoundException. Any reason why this would be
> > happening?
> > > It seems to me like the tasktrackers cannot find the class files from
> the
> > > hadoop program but when you do "hadoop jar", it will automatically ship
> > the
> > > job jar file to all the tasktracker nodes and put them in the
> classpath?
> > >
> > > $ *bin/hadoop jar hadoop-0.20.0-test.jar
> > loadgen*-Dtest.randomtextwrite.bytes_per_map=1048576
> > > -Dhadoop.sort.reduce.keep.percent=50.0 -outKey
> org.apache.hadoop.io.Text
> > > -outValue org.apache.hadoop.io.Text
> > >
> > > No input path; ignoring InputFormat
> > > Job started: Tue Jun 09 03:16:30 PDT 2009
> > > 09/06/09 03:16:30 INFO mapred.JobClient: Running job:
> > job_200906090316_0001
> > > 09/06/09 03:16:31 INFO mapred.JobClient:  map 0% reduce 0%
> > > 09/06/09 03:16:40 INFO mapred.JobClient: Task Id :
> > > attempt_200906090316_0001_m_000000_0, Status : FAILED
> > > java.io.IOException: Split class
> > >
> >
> org.apache.hadoop.mapred.GenericMRLoadGenerator$IndirectInputFormat$IndirectSplit
> > > not found
> > >     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:324)
> > >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> > >     at org.apache.hadoop.mapred.Child.main(Child.java:170)
> > > *Caused by: java.lang.ClassNotFoundException:
> > >
> >
> org.apache.hadoop.mapred.GenericMRLoadGenerator$IndirectInputFormat$IndirectSplit
> > > *
> > >     at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> > >     at java.security.AccessController.doPrivileged(Native Method)
> > >     at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> > >     at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > >     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > >     at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
> > >     at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
> > >     at java.lang.Class.forName0(Native Method)
> > >     at java.lang.Class.forName(Class.java:247)
> > >     at
> > >
> >
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:761)
> > >     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:321)
> > >     ... 2 more
> > > [more of the same exception from different map tasks...I've removed
> them]
> > >
> > > Actually the same thing happens even if I run the PiEstimator program
> > from
> > > hadoop-0.20.0-examples.jar.
> > >
> > > Thanks,
> > >
> > > --
> > > Harish Mallipeddi
> > > http://blog.poundbang.in
> > >
> >
> >
> >
> > --
> > Harish Mallipeddi
> > http://blog.poundbang.in
> >
>



-- 
Harish Mallipeddi
http://blog.poundbang.in

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message