hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shushant Arora <shushantaror...@gmail.com>
Subject Re: bug in hive
Date Tue, 23 Sep 2014 19:24:30 GMT
Hi Alan
1.When writer takes exclusive lock , hive won't allow to write anyone(even
the session which holds lock) to write in table.
Do I  need to pass lock handle to read query or I am missing here something.
2.Or you mean to insert using hadoop filesystem not using hive ?

On Tue, Sep 23, 2014 at 8:13 PM, Alan Gates <gates@hortonworks.com> wrote:

> Shushant,
>
> Creating a patched jar that would include the lock functionality you want
> is unlikely to work.  Wouldn't the following workflow work for you:
>
> 1. Writer locks the table explicitly via LOCK TABLE
> 2. Writer inserts
> 3. Writer unlocks the table explicitly via UNLOCK TABLE
>
> If you're using ZK for your locking I think the client dying (as opposed
> to ending the session) should cause the lock to expire.  If not, you may
> have to assure the unlock happens in your application.  Hope that helps.
>
> Alan.
>
>   Shushant Arora <shushantarora09@gmail.com>
>  September 20, 2014 at 8:00
> Hi Alan
>
> I have 0.10 version of hive deployed in my org's cluster, I cannot update
> that because of org's policy.
> How can I achieve exclusive lock functionality while inserting in dynamic
> partition on hive 0.10 ?
> Does calling hive scripts via some sort of java api with patched jar
> included will help ?
> Moreover hive does not release locks in 0.10 when hive session is killed .
> User has to explicitly unlock a table.
> Can i specify any sort of max expiry time while taking a lock.
>
> Thanks
> Shushant
>
>
>   Alan Gates <gates@hortonworks.com>
>  September 20, 2014 at 7:41
>  Up until Hive 0.13 locks in Hive were really advisory only, since as you
> note any user can remove any other user's lock.  In Hive 0.13 a new type of
> locking was introduced, see
> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-LockManager
> This new locking is automatic and ignores both LOCK and UNLOCK commands.
> Note that it is off by default, you have to configure Hive to use the new
> DbTxnManager to get turn on this locking.  In 0.13 it still has the bug you
> describe as far as acquiring the wrong lock for dynamic partitioning, but I
> believe I've fixed that in 0.14.
>
> Alan.
>
>
>   Shushant Arora <shushantarora09@gmail.com>
>  September 20, 2014 at 5:39
>
> Hive version 0.9 and later has a bug
>
>
>
> While inserting in a hive table Hive takes an exclusive lock. But if table
> is partitioned , and insert is in dynamic partition , it will take shared
> lock on table but if all partitions are static then hive takes exclusive
> lock on partitions in which data is being inserted
>
> and shared lock on table.
>
> https://issues.apache.org/jira/browse/HIVE-3509
>
>
> 1.What if I want to take exclusive lock on table while inserting in
> dynamic partition ?
>
>
> I tried to take explicit lock using :
>
> LOCK TABLE tablename EXCLUSIVE;
>
>
> But it made table to be disabled.
>
> I cannot even read from table anymore even is same session until I do
>
> unlock table tablename in another session;
>
>
> 2. moreover whats lock level in hive , I mean any user can remove any
> other users lock. that too seems buggy.
>
>
> Thanks
>
> Shushant
>
>
>
>
> --
> Sent with Postbox <http://www.getpostbox.com>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Mime
View raw message