hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13929) "org.apache.hadoop.hive.metastore.api.DataOperationType" class not found error when a job is submitted by hive
Date Fri, 03 Jun 2016 01:53:59 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313464#comment-15313464
] 

Ashutosh Chauhan commented on HIVE-13929:
-----------------------------------------

+1

> "org.apache.hadoop.hive.metastore.api.DataOperationType" class not found error when a
job is submitted by hive
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-13929
>                 URL: https://issues.apache.org/jira/browse/HIVE-13929
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 1.3.0, 2.1.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>            Priority: Blocker
>         Attachments: HIVE-13929.patch
>
>
> {noformat}
> 0: jdbc:hive2://os-r6-atlas-ha-re-re-2.openst> create table source1 (abc String);
> No rows affected (0.268 seconds)
> 0: jdbc:hive2://os-r6-atlas-ha-re-re-2.openst> create table ctas_src as select * from
source1;
> INFO  : Tez session hasn't been created yet. Opening session
> INFO  : Dag name: create table ctas_src as select * ...source1(Stage-1)
> INFO  :
> INFO  : Status: Running (Executing on YARN cluster with App id application_1464692782033_0005)
> INFO  : Map 1: -/-
> ERROR : Status: Failed
> ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1464692782033_0005_1_00, diagnostics=[Vertex
vertex_1464692782033_0005_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager,
org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments:
org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator
> at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:70)
> at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:89)
> at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:151)
> at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:148)
> at java..AccessController.doPrivileged(Native Method)
> at javax..auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop..UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.tez.dag.app.dag.RootInputInitializerManager.createInitializer(RootInputInitializerManager.java:148)
> at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInputInitializers(RootInputInitializerManager.java:121)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.setupInputInitializerManager(VertexImpl.java:4607)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.access$4400(VertexImpl.java:202)
> at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.handleInitEvent(VertexImpl.java:3423)
> at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3372)
> at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3353)
> at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1925)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:201)
> at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2053)
> at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2039)
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/DataOperationType
> at org.apache.hadoop.hive.ql.io.AcidUtils$Operation.<clinit>(AcidUtils.java:196)
> at org.apache.hadoop.hive.ql.plan.FileSinkDesc.<init>(FileSinkDesc.java:93)
> at org.apache.hadoop.hive.ql.plan.FileSinkDescConstructorAccess.newInstance(Unknown Source)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo$1.newInstance(Kryo.java:1062)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1112)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSerializer.java:526)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:502)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672)
> at org.apache.hadoop.hive.ql.exec.Utilities.deserializeObjectByKryo(Utilities.java:1173)
> at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:1062)
> at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:1076)
> at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:432)
> at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:311)
> at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.<init>(HiveSplitGenerator.java:101)
> ... 30 more
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.metastore.api.DataOperationType
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 65 more
> ]
> ERROR : DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
> Error: Error while processing statement: FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1464692782033_0005_1_00,
diagnostics=[Vertex vertex_1464692782033_0005_1_00 [Map 1] killed/failed due to:INIT_FAILURE,
Fail to create InputInitializerManager, org.apache.tez.dag.api.TezReflectionException: Unable
to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator
> at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:70)
> at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:89)
> at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:151)
> at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:148)
> at java..AccessController.doPrivileged(Native Method)
> at javax..auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop..UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at org.apache.tez.dag.app.dag.RootInputInitializerManager.createInitializer(RootInputInitializerManager.java:148)
> at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInputInitializers(RootInputInitializerManager.java:121)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.setupInputInitializerManager(VertexImpl.java:4607)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.access$4400(VertexImpl.java:202)
> at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.handleInitEvent(VertexImpl.java:3423)
> at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3372)
> at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3353)
> at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1925)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:201)
> at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2053)
> at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2039)
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
> ... 25 more
> Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/DataOperationType
> at org.apache.hadoop.hive.ql.io.AcidUtils$Operation.<clinit>(AcidUtils.java:196)
> at org.apache.hadoop.hive.ql.plan.FileSinkDesc.<init>(FileSinkDesc.java:93)
> at org.apache.hadoop.hive.ql.plan.FileSinkDescConstructorAccess.newInstance(Unknown Source)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo$1.newInstance(Kryo.java:1062)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1112)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSerializer.java:526)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:502)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:112)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:776)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:139)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:17)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:694)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:507)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672)
> at org.apache.hadoop.hive.ql.exec.Utilities.deserializeObjectByKryo(Utilities.java:1173)
> at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:1062)
> at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:1076)
> at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:432)
> at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:311)
> at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.<init>(HiveSplitGenerator.java:101)
> ... 30 more
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.metastore.api.DataOperationType
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 65 more
> ]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0 (state=08S01,code=2)
> 0: jdbc:hive2://os-r6-atlas-ha-re-re-2.openst>
> {noformat}
> This is the key
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/DataOperationType
> at org.apache.hadoop.hive.ql.io.AcidUtils$Operation.<clinit>(AcidUtils.java:196)
> at org.apache.hadoop.hive.ql.plan.FileSinkDesc.<init>(FileSinkDesc.java:93)
> at org.apache.hadoop.hive.ql.plan.FileSinkDescConstructorAccess.newInstance(Unknown Source)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo$1.newInstance(Kryo.java:1062)
> at org.apache.hive.com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1112)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message