hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11948) Investigate TxnHandler and CompactionTxnHandler to see where we improve concurrency
Date Tue, 17 Nov 2015 17:53:11 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009126#comment-15009126
] 

Alan Gates commented on HIVE-11948:
-----------------------------------

bq. There is a RB link for this patch, it may be easier to add comments there.
I know, I just don't like review board.  It's harder to keep track of comments and feedback
because they're not in the JIRA.  I look forward to moving to pull requests someday so feedback
and changes can be tracked in one place conveniently.

bq. The change in TxnHandler.checkLock() regarding heratbeat is intentional...
Makes sense.

bq. TxnHander unlock(), around line 581...
My mistake, I was thinking that commit() called unlock(), but looking at the code I see it
does the unlocking itself, so only worrying about locks that are not associated with a txn
here makes sense.

bq. Since "dbProduct" is only set once per MS start I don't think this causes any more connections
to be taken out...
That's not what I was saying.  I was saying for programmer convenience it would make sense
to change this to be:
{code}
determineDatabaseProduct(null);
{code}
and change determineDataProduct to be:
{code}
  protected DatabaseProduct determineDatabaseProduct(Connection conn) throws MetaException
{
    if (dbProduct == null) {
      if (conn == null) {
        // get connection...
      }
      try {
      ...
{code}
This is purely convenience and not necessary.

+1 for the patch.

> Investigate TxnHandler and CompactionTxnHandler to see where we improve concurrency
> -----------------------------------------------------------------------------------
>
>                 Key: HIVE-11948
>                 URL: https://issues.apache.org/jira/browse/HIVE-11948
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 0.14.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>         Attachments: HIVE-11948.3.patch, HIVE-11948.4.patch, HIVE-11948.5.patch, HIVE-11948.6.patch,
HIVE-11948.7.patch, HIVE-11948.patch
>
>
> at least some operations (or parts of operations) can run at READ_COMMITTED.
> CompactionTxnHandler.setRunAs()
> CompactionTxnHandler.findNextToCompact()
> if update stmt includes cq_state = '" + INITIATED_STATE + "'" in WHERE clause and logic
to look for "next" candidate
> CompactionTxnHandler.markCompacted()
> perhaps add cq_state=WORKING_STATE in Where clause (mostly as an extra consistency check)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message