hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikkel Kamstrup Erlandsen <...@statsbiblioteket.dk>
Subject Custom Writables in Hadoop RPC
Date Fri, 18 Aug 2006 08:47:31 GMT
Im using a custom Writable in a proxy server feeding records to
hadoop.mapred.RecordReaders. The proxy talks hadoop rpc with the record
reader.

This works fine when I run with mapred.job.tracker="local", but I get a
ClassNotFoundException when i run distributed:

282106-2006-08-17 15:02:34,609 INFO org.apache.hadoop.ipc.Client: Client
connection to 127.0.0.1:19093 caught: java.lang.RuntimeException:
readObject can't find class
282266-java.lang.RuntimeException: readObject can't find class
282322- at
org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:178)
282398- at
org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:60)
282473- at org.apache.hadoop.ipc.Client$Connection.run(Client.java:170)
282538:Caused by: java.lang.ClassNotFoundException:
dk.statsbiblioteket.sbandex.mapred.WritableRecord
282633- at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
282692- at java.security.AccessController.doPrivileged(Native Method)
282755- at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
282818- at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
282876- at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
282942- at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
283000- at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
283066- at java.lang.Class.forName0(Native Method)
283110- at java.lang.Class.forName(Class.java:242)
283154- at
org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:174)
283230- ... 2 more


Do I have to add the module containing
dk.statsbiblioteket.sbandex.mapred.WritableRecord to Hadoops classpath?
I would assume that "hadoop jar myPackage.jar" would import everything
from myPackage.jar...

Cheers,
Mikkel


Mime
View raw message