hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmed Radwan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4652) ValueAggregatorJob sets the wrong job jar
Date Thu, 13 Sep 2012 00:23:08 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454510#comment-13454510
] 

Ahmed Radwan commented on MAPREDUCE-4652:
-----------------------------------------

The issue here is that hadoop-examples.jar is missing from the task's classpath since ValueAggregatorJob
sets its own jar as the job jar, so hadoop-core.jar is used instead of hadoop-examples.jar.
A workaround could be to explicitly add hadoop-examples.jar to the task's classpath via "-libjars"
argument or the "user.jar.file" property. I'll be uploading a patch momentarily which fixes
this default behavior of ValueAggregatorJob.
                
> ValueAggregatorJob sets the wrong job jar
> -----------------------------------------
>
>                 Key: MAPREDUCE-4652
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4652
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: examples, mrv1
>    Affects Versions: 1.0.3
>            Reporter: Ahmed Radwan
>            Assignee: Ahmed Radwan
>
> Using branch-1 tarball, if the user tries to submit an example aggregatewordcount, the
job fails with the following error:
> {code}
> ahmed@ubuntu:~/demo/deploy/hadoop-1.2.0-SNAPSHOT$ bin/hadoop jar hadoop-examples-1.2.0-SNAPSHOT.jar
aggregatewordcount input examples-output/aggregatewordcount 2 textinputformat
> 12/09/12 17:09:46 INFO mapred.JobClient: originalJarPath: /home/ahmed/demo/deploy/hadoop-1.2.0-SNAPSHOT/hadoop-core-1.2.0-SNAPSHOT.jar
> 12/09/12 17:09:48 INFO mapred.JobClient: submitJarFile: hdfs://localhost:9000/tmp/hadoop-ahmed/mapred/staging/ahmed/.staging/job_201209121702_0008/job.jar
> 12/09/12 17:09:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
> 12/09/12 17:09:48 WARN snappy.LoadSnappy: Snappy native library not loaded
> 12/09/12 17:09:48 INFO mapred.FileInputFormat: Total input paths to process : 21
> 12/09/12 17:09:49 INFO mapred.JobClient: Running job: job_201209121702_0008
> 12/09/12 17:09:50 INFO mapred.JobClient:  map 0% reduce 0%
> 12/09/12 17:09:58 INFO mapred.JobClient: Task Id : attempt_201209121702_0008_m_000000_0,
Status : FAILED
> java.lang.RuntimeException: Error in configuring object
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> 	... 9 more
> Caused by: java.lang.RuntimeException: Error in configuring object
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> 	at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> 	... 14 more
> Caused by: java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> 	... 17 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
> 	at org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.createInstance(UserDefinedValueAggregatorDescriptor.java:57)
> 	at org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.createAggregator(UserDefinedValueAggregatorDescriptor.java:64)
> 	at org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.<init>(UserDefinedValueAggregatorDescriptor.java:76)
> 	at org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggregatorDescriptor(ValueAggregatorJobBase.java:54)
> 	at org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorDescriptors(ValueAggregatorJobBase.java:65)
> 	at org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySpec(ValueAggregatorJobBase.java:74)
> 	at org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(ValueAggregatorJobBase.java:42)
> 	... 22 more
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.examples.AggregateWordCount$WordCountPlugInClass
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> 	at java.lang.Class.forName0(Native Method)
> 	at java.lang.Class.forName(Class.java:247)
> 	at org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.createInstance(UserDefinedValueAggregatorDescriptor.java:52)
> 	... 28 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message