hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Furcy Pin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7483) hive insert overwrite table select from self dead lock
Date Wed, 26 Aug 2015 16:34:46 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14714475#comment-14714475
] 

Furcy Pin commented on HIVE-7483:
---------------------------------

I confirm, we are having the same issue on Hive 1.1.0, and I reproduced it on 1.2.1

CREATE TABLE test_db.test_table (id INT) 
PARTITIONED BY (part STRING)
STORED AS ORC ;

INSERT INTO TABLE test_db.test_table PARTITION (part="test")
VALUES (1), (2), (3), (4) 
;

SET hive.exec.dynamic.partition.mode=nonstrict ;
SET hive.support.concurrency=true ;

INSERT OVERWRITE TABLE test_db.test_table PARTITION (part)
SELECT * FROM test_db.test_table ;

nothing happens, and when doing a SHOW LOCKS in another shell we get :

+----------+-----------+------------+------------+-------------+--------------+-----------------+-----------------+----------------+
| lockid   | database  | table      | partition  | lock_state  | lock_type    | transaction_id
 | last_heartbeat  |  acquired_at   |
+----------+-----------+------------+------------+-------------+--------------+-----------------+-----------------+----------------+
| 3765     | test_db   | test_table | NULL       | WAITING     | EXCLUSIVE    | NULL     
      | 1440603633148   | NULL           |
| 3765     | test_db   | test_table | part=test  | WAITING     | SHARED_READ  | NULL     
      | 1440603633148   | NULL           |
+----------+-----------+------------+------------+-------------+--------------+-----------------+-----------------+----------------+

from looking at the source of org.apache.hadoop.hive.ql.lockmgr.EmbeddedLockManager, I would
say it is stuck
in the for loop of the method lock(List<HiveLockObj> objs, int numRetriesForLock, long
sleepTime), 
where lockPrimitive() keeps failing on the second lock.







> hive insert overwrite table select from self dead lock
> ------------------------------------------------------
>
>                 Key: HIVE-7483
>                 URL: https://issues.apache.org/jira/browse/HIVE-7483
>             Project: Hive
>          Issue Type: Bug
>          Components: Locking
>    Affects Versions: 0.13.1
>            Reporter: Xiaoyu Wang
>
> CREATE TABLE test(
>   id int, 
>   msg string)
> PARTITIONED BY ( 
>   continent string, 
>   country string)
> CLUSTERED BY (id) 
> INTO 10 BUCKETS
> STORED AS ORC;
> alter table test add partition(continent='Asia',country='India');
> in hive-site.xml:
> hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> hive.support.concurrency=true;
> in hive shell:
> set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
> insert into test table some records first.
> then execute sql:
> insert overwrite table test partition(continent='Asia',country='India') select id,msg
from test;
> the log stop at :
> INFO log.PerfLogger: <PERFLOG method=acquireReadWriteLocks from=org.apache.hadoop.hive.ql.Driver>
> i think it has dead lock when insert overwrite table from it self.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message