hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HADOOP-927) MapReduce is Broken for User-Defined Classes
Date Thu, 15 Feb 2007 21:52:05 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Doug Cutting resolved HADOOP-927.

    Resolution: Duplicate

> MapReduce is Broken for User-Defined Classes
> --------------------------------------------
>                 Key: HADOOP-927
>                 URL: https://issues.apache.org/jira/browse/HADOOP-927
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.11.0
>            Reporter: Albert Chern
>            Priority: Critical
> Checked out trunk today and all of my jobs with a user-defined class as a map output
value class fail with a ClassNotFoundException.  Here's the stack trace:
> java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> 	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:344)
> 	at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:406)
> 	at org.apache.hadoop.mapred.ReduceTaskRunner.(ReduceTaskRunner.java:339)
> 	at org.apache.hadoop.mapred.ReduceTask.createRunner(ReduceTask.java:91)
> 	at org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:983)
> 	at org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:366)
> 	at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:359)
> 	at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:823)
> 	at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:510)
> 	at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:852)
> 	at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:1494)
> I haven't looked deeply into it, but I would guess that the problem is in the constructor
of ReduceTaskRunner where it attempts to create a SequenceFile.Sorter for the map output value
class.  It's probably done in the main TaskTracker JVM process, so external libraries are
not on its classpath.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message