hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Grover <mgro...@oanda.com>
Subject Exception related to adding a jar in Hive
Date Fri, 23 Dec 2011 18:58:51 GMT
Hi all,
I would really appreciate your thoughts and comments on this.

I am using Hive 0.7.1 and Hadoop 0.20.203 on Amazon Elastic MapReduce.

My .hiverc file looks like:
add jar /home/hadoop/hive/lib/hive-contrib-0.7.1.jar
add jar my_dir/my_jar.jar

However, with the "add jar my_dir/my_jar.jar" in my .hiverc, the Hive CLI is unable to access
the metastore.
When I do, "show tables", I get the following on my CLI:
FAILED: Error in metadata: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

(See the complete exception from Hive log at the end of this email).

If I take out the 2nd add jar line, the Hive CLI works as expected. Afterwards, if I type
the add jar command in CLI (by typing add jar my_dir/my_jar.jar), it successfully gets added
and show tables command returns successfully. Also, this only started happening when I upgraded
to Hadoop 0.20.203 from 0.20.2 (Hive version remains same at 0.7.1). I am still able to use
the .hiverc with the 2nd add jar line successfully with the older hadoop version.

Does anyone have any thoughts on why this could be happening? Below is the complete exception
from the Hive log. Thank you in advance!

NestedThrowables:
java.lang.reflect.InvocationTargetException
org.apache.hadoop.hive.ql.metadata.HiveException: javax.jdo.JDOFatalInternalException: Unexpected
exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
	at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1033)
	at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1018)
	at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:1704)
	at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:296)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:171)
	at org.apache.hadoop.hive.cli.CliDriver.processLineInternal(CliDriver.java:253)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:234)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:487)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
	at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1186)
	at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
	at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
	at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:234)
	at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:261)
	at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:196)
	at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:171)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:354)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:306)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:451)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:232)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:197)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:108)
	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:1914)
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:1924)
	at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1029)
	... 17 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at javax.jdo.JDOHelper$16.run(JDOHelper.java:1958)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.jdo.JDOHelper.invoke(JDOHelper.java:1953)
	at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
	... 34 more
Caused by: java.lang.NullPointerException
	at org.datanucleus.plugin.NonManagedPluginRegistry.registerBundle(NonManagedPluginRegistry.java:443)
	at org.datanucleus.plugin.NonManagedPluginRegistry.registerBundle(NonManagedPluginRegistry.java:355)
	at org.datanucleus.plugin.NonManagedPluginRegistry.registerExtensions(NonManagedPluginRegistry.java:215)
	at org.datanucleus.plugin.NonManagedPluginRegistry.registerExtensionPoints(NonManagedPluginRegistry.java:156)
	at org.datanucleus.plugin.PluginManager.registerExtensionPoints(PluginManager.java:82)
	at org.datanucleus.OMFContext.<init>(OMFContext.java:156)
	at org.datanucleus.OMFContext.<init>(OMFContext.java:137)
	at org.datanucleus.ObjectManagerFactoryImpl.initialiseOMFContext(ObjectManagerFactoryImpl.java:132)
	at org.datanucleus.jdo.JDOPersistenceManagerFactory.initialiseProperties(JDOPersistenceManagerFactory.java:363)
	at org.datanucleus.jdo.JDOPersistenceManagerFactory.<init>(JDOPersistenceManagerFactory.java:307)
	at org.datanucleus.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:255)
	at org.datanucleus.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:182)
	... 42 more

2011-12-23 18:04:03,263 DEBUG hive.ql.exec.DDLTask (DDLTask.java:execute(368)) - org.apache.hadoop.hive.ql.metadata.HiveException:
javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
	at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1033)
	at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1018)
	at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:1704)
	at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:296)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:171)
	at org.apache.hadoop.hive.cli.CliDriver.processLineInternal(CliDriver.java:253)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:234)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:487)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: javax.jdo.JDOFatalInternalException: Unexpected exception caught.
NestedThrowables:
java.lang.reflect.InvocationTargetException
	at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1186)
	at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:803)
	at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:698)
	at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:234)
	at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:261)
	at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:196)
	at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:171)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:354)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.executeWithRetry(HiveMetaStore.java:306)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:451)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:232)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:197)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:108)
	at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:1914)
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:1924)
	at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1029)
	... 17 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at javax.jdo.JDOHelper$16.run(JDOHelper.java:1958)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.jdo.JDOHelper.invoke(JDOHelper.java:1953)
	at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1159)
	... 34 more
Caused by: java.lang.NullPointerException
	at org.datanucleus.plugin.NonManagedPluginRegistry.registerBundle(NonManagedPluginRegistry.java:443)
	at org.datanucleus.plugin.NonManagedPluginRegistry.registerBundle(NonManagedPluginRegistry.java:355)
	at org.datanucleus.plugin.NonManagedPluginRegistry.registerExtensions(NonManagedPluginRegistry.java:215)
	at org.datanucleus.plugin.NonManagedPluginRegistry.registerExtensionPoints(NonManagedPluginRegistry.java:156)
	at org.datanucleus.plugin.PluginManager.registerExtensionPoints(PluginManager.java:82)
	at org.datanucleus.OMFContext.<init>(OMFContext.java:156)
	at org.datanucleus.OMFContext.<init>(OMFContext.java:137)
	at org.datanucleus.ObjectManagerFactoryImpl.initialiseOMFContext(ObjectManagerFactoryImpl.java:132)
	at org.datanucleus.jdo.JDOPersistenceManagerFactory.initialiseProperties(JDOPersistenceManagerFactory.java:363)
	at org.datanucleus.jdo.JDOPersistenceManagerFactory.<init>(JDOPersistenceManagerFactory.java:307)
	at org.datanucleus.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:255)
	at org.datanucleus.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:182)
	... 42 more

2011-12-23 18:04:03,264 ERROR org.apache.hadoop.hive.ql.Driver (SessionState.java:printError(343))
- FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
2011-12-23 18:04:43,309 DEBUG org.apache.hadoop.fs.FileSystem (FileSystem.java:closeAll(265))
- Starting clear of FileSystem cache with 1 elements.
2011-12-23 18:04:43,312 DEBUG org.apache.hadoop.fs.FileSystem (FileSystem.java:close(1196))
- Removing filesystem for file:///
2011-12-23 18:04:43,312 DEBUG org.apache.hadoop.fs.FileSystem (FileSystem.java:close(1196))
- Removing filesystem for file:///
2011-12-23 18:04:43,312 DEBUG org.apache.hadoop.fs.FileSystem (FileSystem.java:closeAll(268))
- Done clearing cache



Mime
View raw message