hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Dere (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17482) External LLAP client: acquire locks for tables queried directly by LLAP
Date Thu, 14 Sep 2017 00:41:03 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165540#comment-16165540
] 

Jason Dere commented on HIVE-17482:
-----------------------------------

So this has not been possible test using a unit test - will test out on a cluster.
Thanks for pointing out that a new transaction is opened during Driver.compile(), that was
not being accounted for in the patch and you're right that the new TxnManager would not have
worked correctly because of that. Reworking the patch so that the Driver class can be initialized
with the TxnManager so that it does not have to use the one from the SessionState. Though
the default behavior will be to use the SessionState txn manager as before.
The configuration will not be clobbered by the next query - each query will get a new Configuration
which will then be passed along in the LLAP input splits.

> External LLAP client: acquire locks for tables queried directly by LLAP
> -----------------------------------------------------------------------
>
>                 Key: HIVE-17482
>                 URL: https://issues.apache.org/jira/browse/HIVE-17482
>             Project: Hive
>          Issue Type: Sub-task
>          Components: llap
>            Reporter: Jason Dere
>            Assignee: Jason Dere
>         Attachments: HIVE-17482.1.patch
>
>
> When using the LLAP external client with simple queries (filter/project of single table),
the appropriate locks should be taken on the table being read like they are for normal Hive
queries. This is important in the case of transactional tables being queried, since the compactor
relies on the presence of table locks to determine whether it can safely delete old versions
of compacted files without affecting currently running queries.
> This does not have to happen in the complex query case, since a query is used (with the
appropriate locking mechanisms) to create/populate the temp table holding the results to the
complex query.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message