hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20921) Oracle backed DbLockManager fail when drop/truncate acid table with large partitions
Date Thu, 15 Nov 2018 07:34:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687595#comment-16687595
] 

Daniel Dai commented on HIVE-20921:
-----------------------------------

Patch for branch-2 attached. On branch-3/master, HIVE-20080's clone HIVE-20518 is not committed.
We need to bring HIVE-20518 to branch-3/master, then apply this patch.

> Oracle backed DbLockManager fail when drop/truncate acid table with large partitions
> ------------------------------------------------------------------------------------
>
>                 Key: HIVE-20921
>                 URL: https://issues.apache.org/jira/browse/HIVE-20921
>             Project: Hive
>          Issue Type: Bug
>          Components: Locking
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>            Priority: Major
>         Attachments: HIVE-20921.branch-2.1.patch
>
>
> Stack:
> {code}
> org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED:
Error in acquiring locks: Error communicating with the metastore 
> at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:324)

> at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:199)

> at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)

> at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)

> at java.security.AccessController.doPrivileged(Native Method) 
> at javax.security.auth.Subject.doAs(Subject.java:422) 
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266) 
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
> at java.lang.Thread.run(Thread.java:745) 
> Caused by: org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with
the metastore 
> at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:177) 
> at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:357)

> at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocksWithHeartbeatDelay(DbTxnManager.java:373)

> at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:182)

> at org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:1082) 
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1284) 
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161) 
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1156) 
> at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)

> ... 11 more 
> Caused by: MetaException(message:How did we get here, we heartbeated our lock before
we started! ( lockid:466073 intLockId:701 txnid:0 db:v5x2442 table:tbstcnf_load_stg_step partition:src_system_cd=MAXIMO/src_hostname_cd=PRD1310/src_table_name=LABTRANS
state:WAITING type:EXCLUSIVE)) 
> at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:2642) 
> at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:1187) 
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.check_lock(HiveMetaStore.java:6161)

> at sun.reflect.GeneratedMethodAccessor135.invoke(Unknown Source) 
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

> at java.lang.reflect.Method.invoke(Method.java:497) 
> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)

> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)

> at com.sun.proxy.$Proxy14.check_lock(Unknown Source) 
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.checkLock(HiveMetaStoreClient.java:1984)

> at sun.reflect.GeneratedMethodAccessor134.invoke(Unknown Source) 
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

> at java.lang.reflect.Method.invoke(Method.java:497) 
> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:178)

> at com.sun.proxy.$Proxy15.checkLock(Unknown Source) 
> at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:114) 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message