hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <>
Subject [jira] [Commented] (HIVE-14350) Aborted txns cause false positive "Not enough history available..." msgs
Date Thu, 28 Jul 2016 22:37:20 GMT


Alan Gates commented on HIVE-14350:

You refactored ValidCompactorTxnList's constructor to trim down the size of the exception
list so that it ends at minOpen.  In the past TxnUtils.createValidCompactTxnList did this,
and it appears that it still does.  Can't that piece be removed from createValidCompactTxnList

> Aborted txns cause false positive "Not enough history available..." msgs
> ------------------------------------------------------------------------
>                 Key: HIVE-14350
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 1.3.0, 2.1.1
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>            Priority: Critical
>         Attachments: HIVE-14350.2.patch, HIVE-14350.3.patch, HIVE-14350.5.patch, HIVE-14350.6.patch,
HIVE-14350.7.patch, HIVE-14350.8.patch, HIVE-14350.9.patch
> this is a followup to HIVE-13369.  Only open txns should prevent use of a base file.
 But ValidTxnList does not make a distinction between open and aborted txns.  The presence
of aborted txns causes false positives which can happen too often since the flow is 
> 1. Worker generates a new base file, 
> 2. then asynchronously Cleaner removes now-compacted aborted txns.  (strictly speaking
it's Initiator that does the actual clean up)
> So we may have base_5 and base_10 and txnid 7 aborted.  Then current impl will disallow
use of base_10 though there is no need for that.  Worse, if txnid_4 is aborted and hasn't
been purged yet, base_5 will be rejected as well and then an error will be raised since there
is no suitable base file left.
> ErrorMsg.ACID_NOT_ENOUGH_HISTORY is msg produced

This message was sent by Atlassian JIRA

View raw message