Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Tue, 17 Nov 2015 16:19:11 +0000 (UTC)
From: "Eugene Koifman (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.12896197.1443119893000.94225.1447777151128@Atlassian.JIRA>
In-Reply-To: <JIRA.12896197.1443119893000@Atlassian.JIRA>
References: <JIRA.12896197.1443119893000@Atlassian.JIRA>
 <JIRA.12896197.1443119893373@arcas>
Subject: [jira] [Commented] (HIVE-11948) Investigate TxnHandler and
 CompactionTxnHandler to see where we improve concurrency
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008937#comment-15008937 ] 

Eugene Koifman commented on HIVE-11948:
---------------------------------------

There is  a RB link for this patch, it may be easier to add comments there.

I'll take care of separate bugs.

The change in TxnHandler.checkLock() regarding heratbeat is intentional.  When there is a txn, only the txn needs to have heartbeat timestamp updated since that is the only timestamp checked for aborting an expired txn.  There is no way/reason to expire a single lock in a txn.  This simplifies both heartbeat and expiration operations (at least when there is a txn).


TxnHander unlock(), around line 581.  The statement to remove the lock is "delete from HIVE_LOCKS where hl_lock_ext_id = " + extLockId + " AND hl_txnid = 0";
The "hl_txnid=0" ensures that this lock doesn't belong to a transaction.  So the unlock() operation runs in  a single SQL statement under most circumstances, but if the above SQL "misses", then there is some additional operations performed to produce a meaningful (best effort) message.  Previously this operation required 3 SQL statements in all cases.

TxnHandler.getRequiredIsolationLevel(), line 2270.  Since "dbProduct" is only set once per MS start I don't think this causes any more connections to be taken out...


> Investigate TxnHandler and CompactionTxnHandler to see where we improve concurrency
> -----------------------------------------------------------------------------------
>
>                 Key: HIVE-11948
>                 URL: https://issues.apache.org/jira/browse/HIVE-11948
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 0.14.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>         Attachments: HIVE-11948.3.patch, HIVE-11948.4.patch, HIVE-11948.5.patch, HIVE-11948.6.patch, HIVE-11948.7.patch, HIVE-11948.patch
>
>
> at least some operations (or parts of operations) can run at READ_COMMITTED.
> CompactionTxnHandler.setRunAs()
> CompactionTxnHandler.findNextToCompact()
> if update stmt includes cq_state = '" + INITIATED_STATE + "'" in WHERE clause and logic to look for "next" candidate
> CompactionTxnHandler.markCompacted()
> perhaps add cq_state=WORKING_STATE in Where clause (mostly as an extra consistency check)


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)