hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <>
Subject [jira] [Commented] (HIVE-11948) Investigate TxnHandler and CompactionTxnHandler to see where we improve concurrency
Date Tue, 17 Nov 2015 16:19:11 GMT


Eugene Koifman commented on HIVE-11948:

There is  a RB link for this patch, it may be easier to add comments there.

I'll take care of separate bugs.

The change in TxnHandler.checkLock() regarding heratbeat is intentional.  When there is a
txn, only the txn needs to have heartbeat timestamp updated since that is the only timestamp
checked for aborting an expired txn.  There is no way/reason to expire a single lock in a
txn.  This simplifies both heartbeat and expiration operations (at least when there is a txn).

TxnHander unlock(), around line 581.  The statement to remove the lock is "delete from HIVE_LOCKS
where hl_lock_ext_id = " + extLockId + " AND hl_txnid = 0";
The "hl_txnid=0" ensures that this lock doesn't belong to a transaction.  So the unlock()
operation runs in  a single SQL statement under most circumstances, but if the above SQL "misses",
then there is some additional operations performed to produce a meaningful (best effort) message.
 Previously this operation required 3 SQL statements in all cases.

TxnHandler.getRequiredIsolationLevel(), line 2270.  Since "dbProduct" is only set once per
MS start I don't think this causes any more connections to be taken out...

> Investigate TxnHandler and CompactionTxnHandler to see where we improve concurrency
> -----------------------------------------------------------------------------------
>                 Key: HIVE-11948
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 0.14.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>         Attachments: HIVE-11948.3.patch, HIVE-11948.4.patch, HIVE-11948.5.patch, HIVE-11948.6.patch,
HIVE-11948.7.patch, HIVE-11948.patch
> at least some operations (or parts of operations) can run at READ_COMMITTED.
> CompactionTxnHandler.setRunAs()
> CompactionTxnHandler.findNextToCompact()
> if update stmt includes cq_state = '" + INITIATED_STATE + "'" in WHERE clause and logic
to look for "next" candidate
> CompactionTxnHandler.markCompacted()
> perhaps add cq_state=WORKING_STATE in Where clause (mostly as an extra consistency check)

This message was sent by Atlassian JIRA

View raw message