hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-13622) WriteSet tracking optimizations
Date Wed, 27 Apr 2016 01:07:12 GMT

     [ https://issues.apache.org/jira/browse/HIVE-13622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eugene Koifman updated HIVE-13622:
----------------------------------
    Description: 
HIVE-13395 solves the the lost update problem with some inefficiencies.

1. TxhHandler.OperationType is currently derived from LockType.  This doesn't  distinguish
between Update and Delete but would be useful.  See comments in TxnHandler.  Should be able
to pass in Insert/Update/Delete info from client into TxnHandler.
2. TxnHandler.addDynamicPartitions() should know the OperationType as well from the client.
 It currently extrapolates it from TXN_COMPONENTS.  This works but requires extra SQL statements
and is thus less performant.  It will not work multi-stmt txns.  See comments in the code.
3. TxnHandler.checkLock() see more comments around "isPartOfDynamicPartitionInsert".  If TxnHandler
knew whether it is being called as part of an op running with dynamic partitions, it could
be more efficient.  In that case we don't have to write to TXN_COMPONENTS at all during lock
acquisition.  Conversely, if not running with DynPart then, we can kill current txn on lock
grant rather than wait until commit time.

All of these require some Thrift changes

Once done, re-enable TestDbTxnHandler2.testWriteSetTracking11()

  was:
HIVE-13395 solves the the lost update problem with some inefficiencies.

1. TxhHandler.OperationType is currently derived from LockType.  This doesn't  distinguish
between Update and Delete but would be useful.  See comments in TxnHandler.  Should be able
to pass in Insert/Update/Delete info from client into TxnHandler.
2. TxnHandler.addDynamicPartitions() should know the OperationType as well from the client.
 It currently extrapolates it from TXN_COMPONENTS.  This works but requires extra SQL statements
and is thus less performant.  It will not work multi-stmt txns.  See comments in the code.
3. TxnHandler.checkLock() see more comments around "isPartOfDynamicPartitionInsert".  If TxnHandler
knew whether it is being called as part of an op running with dynamic partitions, it could
be more efficient.  In that case we don't have to write to TXN_COMPONENTS at all during lock
acquisition.  Conversely, if not running with DynPart then, we can kill current txn on lock
grant rather than wait until commit time.

All of these require some Thrift changes


> WriteSet tracking optimizations
> -------------------------------
>
>                 Key: HIVE-13622
>                 URL: https://issues.apache.org/jira/browse/HIVE-13622
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 1.3.0, 2.1.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>
> HIVE-13395 solves the the lost update problem with some inefficiencies.
> 1. TxhHandler.OperationType is currently derived from LockType.  This doesn't  distinguish
between Update and Delete but would be useful.  See comments in TxnHandler.  Should be able
to pass in Insert/Update/Delete info from client into TxnHandler.
> 2. TxnHandler.addDynamicPartitions() should know the OperationType as well from the client.
 It currently extrapolates it from TXN_COMPONENTS.  This works but requires extra SQL statements
and is thus less performant.  It will not work multi-stmt txns.  See comments in the code.
> 3. TxnHandler.checkLock() see more comments around "isPartOfDynamicPartitionInsert".
 If TxnHandler knew whether it is being called as part of an op running with dynamic partitions,
it could be more efficient.  In that case we don't have to write to TXN_COMPONENTS at all
during lock acquisition.  Conversely, if not running with DynPart then, we can kill current
txn on lock grant rather than wait until commit time.
> All of these require some Thrift changes
> Once done, re-enable TestDbTxnHandler2.testWriteSetTracking11()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message