hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-21757) ACID: use a new write id for compaction's output instead of the visibility id
Date Tue, 21 May 2019 06:27:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-21757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16844553#comment-16844553
] 

Todd Lipcon commented on HIVE-21757:
------------------------------------

Can we introduce some other table-level marker indicating that a compaction has run, then?
We need something to be able to safely cache file lists, and the global list of committed
txns isn't useful for that, considering it changes on every query.

> ACID: use a new write id for compaction's output instead of the visibility id
> -----------------------------------------------------------------------------
>
>                 Key: HIVE-21757
>                 URL: https://issues.apache.org/jira/browse/HIVE-21757
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 4.0.0
>            Reporter: Vaibhav Gumashta
>            Priority: Major
>
> HIVE-20823 added support for running compaction within a transaction. To control the
visibility of the output directory, it uses base_writeId_visibilityId, where visibilityId
is the transaction id of the transaction that the compactor ran in. Perhaps we can keep using
the base_writeId format, by allocating a new writeId for the compactor and creating the new
base/delta with that.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message