hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13836) DbNotifications giving an error = Invalid state. Transaction has already started
Date Thu, 26 May 2016 18:34:13 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302652#comment-15302652
] 

Sushanth Sowmyan commented on HIVE-13836:
-----------------------------------------

Thanks for the catch, [~vaidyand].

I have a couple of thoughts - firstly, since the notion of a Notification Log Table is dependent
on the RawStore implementation, the lock might belong more in implementations of RawStore
such as ObjectStore, rather than in DbNotificationListener.

Secondly, when I went and looked at ObjectStore, I see that we're correctly calling openTransaction()/commitTransaction()/rollbackTransaction()
which should serve the same purposes as the lock manages, and if you're getting an error that
states "Transaction has already started", it's likely that we're hitting a deeper bug with
transaction semantics(with nesting allowed for) in ObjectStore. Adding a lock in DbNotificationListener
will fix this bug in DbNotificationListener, but we have some other issue that we're not discovering.

[~alangates], thoughts?

> DbNotifications giving an error = Invalid state. Transaction has already started
> --------------------------------------------------------------------------------
>
>                 Key: HIVE-13836
>                 URL: https://issues.apache.org/jira/browse/HIVE-13836
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Nachiket Vaidya
>            Priority: Critical
>         Attachments: HIVE-13836.patch
>
>
> I used pyhs2 python client to create tables/partitions in hive. I was working fine until
I moved to multithreaded scripts which created 8 connections and ran DDL queries concurrently.
> I got the error as
> {noformat}
> 2016-05-04 17:49:26,226 ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-4-thread-194]:
HMSHandler Fatal error: Invalid state. Transaction has already started
> org.datanucleus.transaction.NucleusTransactionException: Invalid state. Transaction has
already started
>         at org.datanucleus.transaction.TransactionManager.begin(TransactionManager.java:47)
>         at org.datanucleus.TransactionImpl.begin(TransactionImpl.java:131)
>         at org.datanucleus.api.jdo.JDOTransaction.internalBegin(JDOTransaction.java:88)
>         at org.datanucleus.api.jdo.JDOTransaction.begin(JDOTransaction.java:80)
>         at org.apache.hadoop.hive.metastore.ObjectStore.openTransaction(ObjectStore.java:463)
>         at org.apache.hadoop.hive.metastore.ObjectStore.addNotificationEvent(ObjectStore.java:7522)
>         at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
>         at com.sun.proxy.$Proxy10.addNotificationEvent(Unknown Source)
>         at org.apache.hive.hcatalog.listener.DbNotificationListener.enqueue(DbNotificationListener.java:261)
>         at org.apache.hive.hcatalog.listener.DbNotificationListener.onCreateTable(DbNotificationListener.java:123)
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1483)
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1502)
>         at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:138)
>         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
>         at com.sun.proxy.$Proxy14.create_table_with_environment_context(Unknown Source)
>         at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:9267)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message