hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shushant Arora <shushantaror...@gmail.com>
Subject Re: bug in hive
Date Sat, 20 Sep 2014 15:00:36 GMT
Hi Alan

I have 0.10 version of hive deployed in my org's cluster, I cannot update
that because of org's policy.
How can I achieve exclusive lock functionality while inserting in dynamic
partition on hive 0.10 ?
Does calling hive scripts via some sort of java api with patched jar
included will help ?
Moreover hive does not release locks in 0.10 when hive session is killed .
User has to explicitly unlock a table.
Can i specify any sort of max expiry time while taking a lock.

Thanks
Shushant

On Sat, Sep 20, 2014 at 8:11 PM, Alan Gates <gates@hortonworks.com> wrote:

> Up until Hive 0.13 locks in Hive were really advisory only, since as you
> note any user can remove any other user's lock.  In Hive 0.13 a new type of
> locking was introduced, see
> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-LockManager
> This new locking is automatic and ignores both LOCK and UNLOCK commands.
> Note that it is off by default, you have to configure Hive to use the new
> DbTxnManager to get turn on this locking.  In 0.13 it still has the bug you
> describe as far as acquiring the wrong lock for dynamic partitioning, but I
> believe I've fixed that in 0.14.
>
> Alan.
>
>   Shushant Arora <shushantarora09@gmail.com>
>  September 20, 2014 at 5:39
>
> Hive version 0.9 and later has a bug
>
>
>
> While inserting in a hive table Hive takes an exclusive lock. But if table
> is partitioned , and insert is in dynamic partition , it will take shared
> lock on table but if all partitions are static then hive takes exclusive
> lock on partitions in which data is being inserted
>
> and shared lock on table.
>
> https://issues.apache.org/jira/browse/HIVE-3509
>
>
> 1.What if I want to take exclusive lock on table while inserting in
> dynamic partition ?
>
>
> I tried to take explicit lock using :
>
> LOCK TABLE tablename EXCLUSIVE;
>
>
> But it made table to be disabled.
>
> I cannot even read from table anymore even is same session until I do
>
> unlock table tablename in another session;
>
>
> 2. moreover whats lock level in hive , I mean any user can remove any
> other users lock. that too seems buggy.
>
>
> Thanks
>
> Shushant
>
>
>
>
> --
> Sent with Postbox <http://www.getpostbox.com>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Mime
View raw message