hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Albert Chern (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-927) MapReduce is Broken for User-Defined Classes
Date Thu, 15 Feb 2007 21:43:05 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473524

Albert Chern commented on HADOOP-927:

This issue was the same as Hadoop-964 so it can be closed now.

> MapReduce is Broken for User-Defined Classes
> --------------------------------------------
>                 Key: HADOOP-927
>                 URL: https://issues.apache.org/jira/browse/HADOOP-927
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.11.0
>            Reporter: Albert Chern
>            Priority: Critical
> Checked out trunk today and all of my jobs with a user-defined class as a map output
value class fail with a ClassNotFoundException.  Here's the stack trace:
> java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> 	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:344)
> 	at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:406)
> 	at org.apache.hadoop.mapred.ReduceTaskRunner.(ReduceTaskRunner.java:339)
> 	at org.apache.hadoop.mapred.ReduceTask.createRunner(ReduceTask.java:91)
> 	at org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:983)
> 	at org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:366)
> 	at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:359)
> 	at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:823)
> 	at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:510)
> 	at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:852)
> 	at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:1494)
> I haven't looked deeply into it, but I would guess that the problem is in the constructor
of ReduceTaskRunner where it attempts to create a SequenceFile.Sorter for the map output value
class.  It's probably done in the main TaskTracker JVM process, so external libraries are
not on its classpath.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message