hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HADOOP-12) InputFormat used in job must be in JobTracker classpath (not loaded from job JAR)
Date Thu, 09 Feb 2006 23:01:56 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-12?page=all ]
     
Doug Cutting resolved HADOOP-12:
--------------------------------

    Resolution: Fixed

I fixed this by having the JobTracker load the InputFormat from the job's jar.

> InputFormat used in job must be in JobTracker classpath (not loaded from job JAR)
> ---------------------------------------------------------------------------------
>
>          Key: HADOOP-12
>          URL: http://issues.apache.org/jira/browse/HADOOP-12
>      Project: Hadoop
>         Type: Bug
>   Components: mapred
>  Environment: ~20 node nutch mapreduce environment, running SVN trunk, on Linux
>     Reporter: Bryan Pendleton
>     Priority: Minor

>
> During development, I've been creating/tweaking custom InputFormat implementations. However,
when you try to run a job against a running cluster, you get:
>   Exception in thread "main" java.io.IOException: java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: my.custom.InputFormat
>           at org.apache.nutch.ipc.Client.call(Client.java:294)
>           at org.apache.nutch.ipc.RPC$Invoker.invoke(RPC.java:127)
>           at $Proxy0.submitJob(Unknown Source)
>           at org.apache.nutch.mapred.JobClient.submitJob(JobClient.java:259)
>           at org.apache.nutch.mapred.JobClient.runJob(JobClient.java:288)
>           at com.parc.uir.wikipedia.WikipediaJob.main(WikipediaJob.java:85)
> This error goes away if I restart the TaskTrackers/JobTracker with a classpath which
includes the needed code. Other classes (Mapper, Reducer) appear to be available out of the
jar file specified in the JobConf, but not the InputFormat. Obviously, it's less than idea
to have to restart the JobTracker whenever there's a change to a job-specific class.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message