hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerry Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9260) Hadoop version may be not correct when starting name node or data node
Date Wed, 30 Jan 2013 08:01:17 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13566271#comment-13566271
] 

Jerry Chen commented on HADOOP-9260:
------------------------------------

The incorrect version would cause more problems when running MapReduce jobs even if the cluster
seems to start as normal. For example, when running Hive queries, the query failed at random
and the log shows:

2013-01-30 15:45:37,080 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running
child : java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
	at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)
	at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)
	at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:370)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:402)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1452)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:126)
	... 10 more
Caused by: java.lang.NumberFormatException: For input string: "${pom"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
	at java.lang.Integer.parseInt(Integer.java:449)
	at java.lang.Integer.parseInt(Integer.java:499)
	at org.apache.hadoop.hive.shims.ShimLoader.getMajorVersion(ShimLoader.java:125)
	at org.apache.hadoop.hive.shims.ShimLoader.loadShims(ShimLoader.java:92)
	at org.apache.hadoop.hive.shims.ShimLoader.getHadoopShims(ShimLoader.java:66)
	at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat$CombineHiveInputSplit.<init>(CombineHiveInputFormat.java:85)
	... 15 more

                
> Hadoop version may be not correct when starting name node or data node
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-9260
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9260
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: trunk-win
>            Reporter: Jerry Chen
>            Priority: Critical
>
> 1. Check out the trunk from http://svn.apache.org/repos/asf/hadoop/common/trunk/ -r 1439752
> 2. Compile package
>    m2 package -Pdist -Psrc -Pnative -Dtar -DskipTests
> 3. Hadoop version of compiled dist shows the following:
> Hadoop ${pom.version}
> Subversion ${version-info.scm.uri} -r ${version-info.scm.commit}
> Compiled by ${user.name} on ${version-info.build.time}
> From source with checksum ${version-info.source.md5}
> And in a real cluster, the log in name node shows:
> 2013-01-29 15:23:42,738 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG:

> /************************************************************
> STARTUP_MSG: Starting NameNode
> STARTUP_MSG:   host = bdpe01.sh.intel.com/10.239.47.101
> STARTUP_MSG:   args = []
> STARTUP_MSG:   version = ${pom.version}
> STARTUP_MSG:   classpath = ...
> STARTUP_MSG:   build = ${version-info.scm.uri} -r ${version-info.scm.commit}; compiled
by '${user.name}' on ${version-info.build.time}
> STARTUP_MSG:   java = 1.6.0_33
> While some data nodes with the same binary shows the correct version information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message