hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8311) Driver is encoding transaction information too late
Date Wed, 01 Oct 2014 00:25:33 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154090#comment-14154090
] 

Alan Gates commented on HIVE-8311:
----------------------------------

bq. So now it's possible that txn list is computed...
If another txn runs between when a query generates its txn list and when it acquires locks
that's fine because this txn won't see the results of that txn.  I think this is more reasonable
then having the query wait and generate its txn state after locks, because that makes the
results of the query variable based on when locks are obtained instead of when it is submitted.

(Note that this issue exists no matter what.  Even once locks are acquired it may take some
time for the engine (Tez or MR) to submit the job.  Also, once we support cross query transactions
we'll have multiple queries where we are acquiring new sets of locks.)

If the compactor runs between when the query gets its txn list and when it obtains locks we
should still be fine.  By definition the compactor does not change the set of valid data.
 Some txns listed as aborted in the query's list may now be removed from the file, but that's
not a problem.

Have I missed anything?

bq. Does hive cache query plans? 
Not at the moment.

> Driver is encoding transaction information too late
> ---------------------------------------------------
>
>                 Key: HIVE-8311
>                 URL: https://issues.apache.org/jira/browse/HIVE-8311
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 0.14.0
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>            Priority: Blocker
>             Fix For: 0.14.0
>
>         Attachments: HIVE-8311.patch
>
>
> Currently Driver is obtaining the transaction information and encoding it in the conf
in runInternal.  But this is too late, as the query has already been planned.  Either we need
to change the plan when this info is obtained or we need to obtain it at compile time.  This
bug was introduced by HIVE-8203.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message