hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17840) HiveMetaStore eats exception if transactionalListeners.notifyEvent fail
Date Thu, 19 Oct 2017 17:25:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16211385#comment-16211385
] 

Daniel Dai commented on HIVE-17840:
-----------------------------------

Here is the exception HiveMetastore eats:
{code}
javax.jdo.JDODataStoreException: Error executing SQL query "select "NEXT_EVENT_ID" from "NOTIFICATION_SEQUENCE"
for update".
	at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
	at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:388)
	at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:213)
	at org.apache.hadoop.hive.metastore.ObjectStore.lambda$lockForUpdate$0(ObjectStore.java:7360)
	at org.apache.hadoop.hive.metastore.ObjectStore$RetryingExecutor.run(ObjectStore.java:7389)
	at org.apache.hadoop.hive.metastore.ObjectStore.lockForUpdate(ObjectStore.java:7361)
	at org.apache.hadoop.hive.metastore.ObjectStore.addNotificationEvent(ObjectStore.java:7424)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:103)
	at com.sun.proxy.$Proxy10.addNotificationEvent(Unknown Source)
	at org.apache.hive.hcatalog.listener.DbNotificationListener.process(DbNotificationListener.java:481)
	at org.apache.hive.hcatalog.listener.DbNotificationListener.onAddPartition(DbNotificationListener.java:278)
	at org.apache.hadoop.hive.metastore.MetaStoreListenerNotifier$10.notify(MetaStoreListenerNotifier.java:86)
	at org.apache.hadoop.hive.metastore.MetaStoreListenerNotifier.notifyEvent(MetaStoreListenerNotifier.java:166)
	at org.apache.hadoop.hive.metastore.MetaStoreListenerNotifier.notifyEvent(MetaStoreListenerNotifier.java:196)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.add_partition_core(HiveMetaStore.java:2819)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.add_partition_with_environment_context(HiveMetaStore.java:2859)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
	at com.sun.proxy.$Proxy18.add_partition_with_environment_context(Unknown Source)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$add_partition_with_environment_context.getResult(ThriftHiveMetastore.java:9994)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$add_partition_with_environment_context.getResult(ThriftHiveMetastore.java:9978)
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:551)
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:546)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:546)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
{code}
And here is the consequence it leads to:
{code}
	at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:525)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1751)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1497)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1294)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1151)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:217)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:169)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:380)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:315)
	at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:413)
	at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:429)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:718)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:685)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
	at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:336)
	at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:313)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:58)
	at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:69)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:239)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
{code}

> HiveMetaStore eats exception if transactionalListeners.notifyEvent fail
> -----------------------------------------------------------------------
>
>                 Key: HIVE-17840
>                 URL: https://issues.apache.org/jira/browse/HIVE-17840
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>
> For example, in add_partitions_core, if there's exception in MetaStoreListenerNotifier.notifyEvent(transactionalListeners,....),
transaction rollback but no exception thrown. Client will assume add partition is successful
and take a positive path.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message