hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Furcy Pin <furcy....@flaminem.com>
Subject Re: locks are held on tables even when no job running
Date Fri, 23 Oct 2015 08:07:41 GMT
Hi Mich,

I believe the duration of locks is defined by hive.txn.timeout, which is 5
min by default.
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties

Retry your SHOW LOCKS command and check that the Last HeartBeat is not
changing.
If it is, it means your query is still active somehow. If it isn't, the
lock should disappear by itself after the timeout.

Also, I don't know if this is a known issue in Hive's Jira, but we noticed
that trying to read and write data
into the same table within a single query creates a deadlock.

If that is what you were trying to do, you should rather write your data in
a temporary file and then move it back into the table.








On Fri, Oct 23, 2015 at 9:56 AM, Mich Talebzadeh <mich@peridale.co.uk>
wrote:

> Hi,
>
>
>
>
>
> What is the duration of locks held in Hive?
>
>
>
> I have got the following locks in Hive, although I have already killed the
> jobs!
>
>
>
>
>
> Lock ID Database        Table   Partition       State   Type
> Transaction ID  Last Hearbeat   Acquired At     User    Hostname
>
> 14031   asehadoop       t       NULL    WAITING EXCLUSIVE       NULL
> 1445586539232   NULL    hduser  rhes564
>
> 14029   asehadoop       t_staging       NULL    ACQUIRED
> SHARED_READ     NULL    1445586247044   1445585940653   hduser  rhes564
>
> 14029   asehadoop       t       NULL    ACQUIRED        SHARED_READ
> NULL    1445586247044   1445585940654   hduser  rhes564
>
> 14030   asehadoop       t       NULL    WAITING EXCLUSIVE       NULL
> 1445586471827   NULL    hduser  rhes564
>
>
>
>
>
> mapred job -list
>
> Total jobs:0
>
>                   JobId      State           StartTime
> UserName           Queue      Priority       UsedContainers
> RsvdContainers  UsedMem         RsvdMem         NeededMem         AM info
>
>
>
> No locks are held in metastore (Oracle in my case) as well.
>
>
>
> Thanks
>
>
>
>
>
> Mich Talebzadeh
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>

Mime
View raw message