hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <>
Subject [jira] [Commented] (HIVE-13836) DbNotifications giving an error = Invalid state. Transaction has already started
Date Thu, 26 May 2016 18:34:13 GMT


Sushanth Sowmyan commented on HIVE-13836:

Thanks for the catch, [~vaidyand].

I have a couple of thoughts - firstly, since the notion of a Notification Log Table is dependent
on the RawStore implementation, the lock might belong more in implementations of RawStore
such as ObjectStore, rather than in DbNotificationListener.

Secondly, when I went and looked at ObjectStore, I see that we're correctly calling openTransaction()/commitTransaction()/rollbackTransaction()
which should serve the same purposes as the lock manages, and if you're getting an error that
states "Transaction has already started", it's likely that we're hitting a deeper bug with
transaction semantics(with nesting allowed for) in ObjectStore. Adding a lock in DbNotificationListener
will fix this bug in DbNotificationListener, but we have some other issue that we're not discovering.

[~alangates], thoughts?

> DbNotifications giving an error = Invalid state. Transaction has already started
> --------------------------------------------------------------------------------
>                 Key: HIVE-13836
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Nachiket Vaidya
>            Priority: Critical
>         Attachments: HIVE-13836.patch
> I used pyhs2 python client to create tables/partitions in hive. I was working fine until
I moved to multithreaded scripts which created 8 connections and ran DDL queries concurrently.
> I got the error as
> {noformat}
> 2016-05-04 17:49:26,226 ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-4-thread-194]:
HMSHandler Fatal error: Invalid state. Transaction has already started
> org.datanucleus.transaction.NucleusTransactionException: Invalid state. Transaction has
already started
>         at org.datanucleus.transaction.TransactionManager.begin(
>         at org.datanucleus.TransactionImpl.begin(
>         at org.datanucleus.api.jdo.JDOTransaction.internalBegin(
>         at org.datanucleus.api.jdo.JDOTransaction.begin(
>         at org.apache.hadoop.hive.metastore.ObjectStore.openTransaction(
>         at org.apache.hadoop.hive.metastore.ObjectStore.addNotificationEvent(
>         at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>         at java.lang.reflect.Method.invoke(
>         at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(
>         at com.sun.proxy.$Proxy10.addNotificationEvent(Unknown Source)
>         at org.apache.hive.hcatalog.listener.DbNotificationListener.enqueue(
>         at org.apache.hive.hcatalog.listener.DbNotificationListener.onCreateTable(
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(
>         at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>         at java.lang.reflect.Method.invoke(
>         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(
>         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(
>         at com.sun.proxy.$Proxy14.create_table_with_environment_context(Unknown Source)
>         at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(
> {noformat}

This message was sent by Atlassian JIRA

View raw message