Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Wed, 23 Mar 2016 00:33:25 +0000 (UTC)
From: "Wei Zheng (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.12849495.1438046219000.24536.1458693205443@Atlassian.JIRA>
In-Reply-To: <JIRA.12849495.1438046219000@Atlassian.JIRA>
References: <JIRA.12849495.1438046219000@Atlassian.JIRA>
 <JIRA.12849495.1438046219480@arcas>
Subject: [jira] [Commented] (HIVE-11388) Allow ACID Compactor components to
 run in multiple metastores
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-11388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207615#comment-15207615 ] 

Wei Zheng commented on HIVE-11388:
----------------------------------

[~ekoifman] I have several questions regarding patch 7.
1. In TxnHandler.acquireLock implementation, there's a {code}if (!rs.next()){code}block, after that, shouldn't there be an else block that deals with the case when there's existing key in AUX_TABLE (thus roll back the select for update and retry)?
2. In Cleaner.run(), I'm not sure if we need currentToCleanSet, since we're essentially checking the existence of compactId2CompactInfoMap members in toClean set.
3. In TestTxnHandler.testMutexAPI, we can add two more asserts after //now 2 and //now 3 to confirm.

> Allow ACID Compactor components to run in multiple metastores
> -------------------------------------------------------------
>
>                 Key: HIVE-11388
>                 URL: https://issues.apache.org/jira/browse/HIVE-11388
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 1.0.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>            Priority: Critical
>         Attachments: HIVE-11388.2.patch, HIVE-11388.4.patch, HIVE-11388.5.patch, HIVE-11388.6.patch, HIVE-11388.7.patch, HIVE-11388.patch
>
>
> (this description is no loner accurate; see further comments)
> org.apache.hadoop.hive.ql.txn.compactor.Initiator is a thread that runs inside the metastore service to manage compactions of ACID tables.  There should be exactly 1 instance of this thread (even with multiple Thrift services).
> This is documented in https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration but not enforced.
> Should add enforcement, since more than 1 Initiator could cause concurrent attempts to compact the same table/partition - which will not work.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)