hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sankar Hariappan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-18192) Introduce WriteID per table rather than using global transaction ID
Date Wed, 07 Feb 2018 04:19:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354062#comment-16354062
] 

Sankar Hariappan edited comment on HIVE-18192 at 2/7/18 4:18 AM:
-----------------------------------------------------------------

Added 09.patch with below updates.
 # Remove entries from TXNS_TO_WRITE_ID table when drop a table/database. Also, split db and
table names into 2 columns.
 # Multi-statement txn support by setting ValidTxnList for open txn and get/set ValidWriteId
list for each operation within txn.

Pending changes for this feature
 # Scripts to add Metastore tables for write id management. Add for other non-derby databases
too.
 # CompactionTxnHandler cleanup COMPLETED_TXN_COMPONENTS based on highest write id instead
of highest txn id.
 # Address review comments from Eugene.
 # Address few more test failures.
 # Fix check style issues.

Out of scope of HIVE-18192:
 # Cleaner for TXNS_TO_WRITE_ID table entries. Also, need to maintain the LWM for each table
write id.
 # Update classes to use WriteID instead of TxnId as methods and variable names.


was (Author: sankarh):
Added 09.patch with below updates.
 # Remove entries from TXNS_TO_WRITE_ID table when drop a table/database. Also, split db and
table names into 2 columns.
 # Multi-statement txn support by setting ValidTxnList for open txn and get/set ValidWriteId
list for each operation within txn.

Pending changes for this feature
 # Scripts to add Metastore tables for write id management. Add for other non-derby databases
too.
 # CompactionTxnHandler cleanup COMPLETED_TXN_COMPONENTS based on highest write id instead
of highest txn id.
 # Address few more test failures.
 # Fix check style issues.

Out of scope of HIVE-18192:
 # Cleaner for TXNS_TO_WRITE_ID table entries. Also, need to maintain the LWM for each table
write id.
 # Update classes to use WriteID instead of TxnId as methods and variable names.

> Introduce WriteID per table rather than using global transaction ID
> -------------------------------------------------------------------
>
>                 Key: HIVE-18192
>                 URL: https://issues.apache.org/jira/browse/HIVE-18192
>             Project: Hive
>          Issue Type: Sub-task
>          Components: HiveServer2, Transactions
>    Affects Versions: 3.0.0
>            Reporter: anishek
>            Assignee: Sankar Hariappan
>            Priority: Major
>              Labels: ACID, DR, pull-request-available
>             Fix For: 3.0.0
>
>         Attachments: HIVE-18192.01.patch, HIVE-18192.02.patch, HIVE-18192.03.patch, HIVE-18192.04.patch,
HIVE-18192.05.patch, HIVE-18192.06.patch, HIVE-18192.07.patch, HIVE-18192.08.patch, HIVE-18192.09.patch
>
>
> To support ACID replication, we will be introducing a per table write Id which will replace
the transaction id in the primary key for each row in a ACID table.
> The current primary key is determined via 
>  <original transaction id, bucketid , rowid>
> which will move to 
>  <original write id, bucketid, rowid>
> For each table modified by the given transaction will have a table level write ID allocated
and a persisted map of global txn id -> to table -> write id for that table has to
be maintained to allow Snapshot isolation.
> Readers should use the combination of ValidTxnList and ValidWriteIdList(Table) for snapshot
isolation.
>  
>  [Hive Replication - ACID Tables.pdf|https://issues.apache.org/jira/secure/attachment/12903157/Hive%20Replication-%20ACID%20Tables.pdf]
has a section "Per Table Sequences (Write-Id)" with more detials



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message