hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-12389) CompactionTxnHandler.cleanEmptyAbortedTxns() should safeguard against huge IN clauses
Date Thu, 12 Nov 2015 02:46:11 GMT
Eugene Koifman created HIVE-12389:
-------------------------------------

             Summary: CompactionTxnHandler.cleanEmptyAbortedTxns() should safeguard against
huge IN clauses
                 Key: HIVE-12389
                 URL: https://issues.apache.org/jira/browse/HIVE-12389
             Project: Hive
          Issue Type: Bug
          Components: Metastore, Transactions
    Affects Versions: 1.0.0
            Reporter: Eugene Koifman
            Assignee: Eugene Koifman


in extreme situations, due to misconfigurations, it may be possible to have 100Ks or even
1Ms of aborted txns.
This causes delete from TXNS where txn_id in (...) to have a huge IN clause and DB chokes.
 

Should use something like TxnHandler.TIMED_OUT_TXN_ABORT_BATCH_SIZE to break up delete into
multiple queries.  (Incidentally the batch size should likely be 1000, not 100, maybe even
configurable).





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message