hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5160) Aggregatewordcount and aggregatewordhist in hadoop-1 examples can not find their inner classes when running on Yarn
Date Wed, 17 Apr 2013 18:27:17 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634285#comment-13634285
] 

Zhijie Shen commented on MAPREDUCE-5160:
----------------------------------------

The exception observed when running aggregatewordhist:
{code}
2013-04-17 11:04:13,471 FATAL [IPC Server handler 7 on 61283] org.apache.hadoop.mapred.TaskAttemptListenerImpl:
Task: attempt_1366220242098_0004_m_000007_0 - exited : java.lang.RuntimeException: Error in
configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:425)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1489)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
	... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
	at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
	... 14 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
	... 17 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.examples.AggregateWordHistogram$AggregateWordHistogramPlugin
	at org.apache.hadoop.mapreduce.lib.aggregate.UserDefinedValueAggregatorDescriptor.createInstance(UserDefinedValueAggregatorDescriptor.java:63)
	at org.apache.hadoop.mapreduce.lib.aggregate.UserDefinedValueAggregatorDescriptor.createAggregator(UserDefinedValueAggregatorDescriptor.java:70)
	at org.apache.hadoop.mapreduce.lib.aggregate.UserDefinedValueAggregatorDescriptor.<init>(UserDefinedValueAggregatorDescriptor.java:84)
	at org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.<init>(UserDefinedValueAggregatorDescriptor.java:55)
	at org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggregatorDescriptor(ValueAggregatorJobBase.java:58)
	at org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorDescriptors(ValueAggregatorJobBase.java:69)
	at org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySpec(ValueAggregatorJobBase.java:78)
	at org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(ValueAggregatorJobBase.java:46)
	... 22 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.examples.AggregateWordHistogram$AggregateWordHistogramPlugin
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:249)
	at org.apache.hadoop.mapreduce.lib.aggregate.UserDefinedValueAggregatorDescriptor.createInstance(UserDefinedValueAggregatorDescriptor.java:58)
	... 29 more
{code}
                
> Aggregatewordcount and aggregatewordhist in hadoop-1 examples can not find their inner
classes when running on Yarn
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5160
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5160
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>
> Aggregatewordcount and Aggregatewordhist of hadoop-1 cannot run on hadoop-2 due to org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJob.createValueAggregatorJob(String
args[], Class<?> caller) is not available on hadoop-2 (see MAPREDUCE-5159).
> After I changed Aggregatewordcount and Aggregatewordhist to use createValueAggregatorJob(String
args[], Class<? extends ValueAggregatorDescriptor>[] descriptors), which is available
on hadoop-2, the two examples could be accepted and run on Yarn.
> However, the two examples still failed, because their inner classes, WordCountPlugInClass
and AggregateWordHistogramPlugin, cannot be found in runtime, respectively. Both the plugin
classes extend org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorBaseDescriptor.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message