Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 73AFE104DD for ; Thu, 4 Dec 2014 03:56:17 +0000 (UTC) Received: (qmail 8661 invoked by uid 500); 4 Dec 2014 03:56:15 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 8591 invoked by uid 500); 4 Dec 2014 03:56:15 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 8581 invoked by uid 99); 4 Dec 2014 03:56:15 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Dec 2014 03:56:15 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of yogendra.60@gmail.com designates 209.85.192.43 as permitted sender) Received: from [209.85.192.43] (HELO mail-qg0-f43.google.com) (209.85.192.43) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Dec 2014 03:56:11 +0000 Received: by mail-qg0-f43.google.com with SMTP id q108so12225080qgd.30 for ; Wed, 03 Dec 2014 19:55:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=dzK0Ne+6zh9dyBIw0D5HdLXrJ2od5cMO6Hje+Kyo44E=; b=F+zbwqSkG3we+Mg5gdpbc06ilgE5WNmBtr7yh1J6dz7RQV5wh/VSI8v8v8t5h/ztX3 xn6Hq7j63dqHPiw5jCm8H4msEFWuHi1Hbo9BWruJ3oEyghi13KL/sK/1nhozzaJvguqQ PUmYx3RccV3D1JmWOLoTK4u2fkDYnia/t6q1TZSDAZh97rjAt7Bzufm72VtZVfu0bUjZ omGmgAaVdGQI79Mi/vr8scrpdppf34A+LhoXxEWbu1YsMaohsda36RslUCfVPS2M7zE8 IZI0nYuCMXL4E4QDaCugtmL2KdKeq/t3JufXSbZN1AdDBbGl4fk+k4wg2G2M7aT1aU1h ZY8g== MIME-Version: 1.0 X-Received: by 10.224.28.135 with SMTP id m7mr13425272qac.7.1417665305858; Wed, 03 Dec 2014 19:55:05 -0800 (PST) Received: by 10.140.106.68 with HTTP; Wed, 3 Dec 2014 19:55:05 -0800 (PST) In-Reply-To: References: <547E7077.2040705@hortonworks.com> <547fc6db.e481460a.3a23.ffffe396@mx.google.com> Date: Thu, 4 Dec 2014 09:25:05 +0530 Message-ID: Subject: Re: hive transactions From: yogendra reddy To: user@hive.apache.org Content-Type: multipart/related; boundary=001a11c1fca2f76f5b05095bec69 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c1fca2f76f5b05095bec69 Content-Type: multipart/alternative; boundary=001a11c1fca2f76f5605095bec68 --001a11c1fca2f76f5605095bec68 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks Vikas Srivastava, I do have lot of space available in my metastore d= b Thanks, Yogendra On Thu, Dec 4, 2014 at 9:12 AM, hadoop hive wrote: > Hi yogendra, > > You need to check if space is available on you backend metadata db server > like mysql or oracle server. > > It might be possible the your queries are getting stuck becoz there is no > enough space for mysql logging. > > Thanks > Vikas Srivastava > On Dec 4, 2014 7:59 AM, "Yogendra reddy" wrote: > >> Thanks Alan, I'm able to see TXNS table and yes I have upgraded from 0.1= 3 >> to 0.14. >> >> any idea as to wht else might cause this issue? >> >> Thanks, >> Yogendra >> ------------------------------ >> From: Alan Gates >> Sent: =E2=80=8E12/=E2=80=8E3/=E2=80=8E2014 9:51 AM >> To: user@hive.apache.org >> Subject: Re: hive transactions >> >> Usually this means the metastore database was not properly upgraded. Yo= u >> can check this by logging into your RDBMS and listing all the tables. I= f >> you see a table named TXNS then the proper tables exist. >> >> Did you install Hive 0.14 fresh or upgrade from a previous version? If >> you upgraded did you run the upgrade scripts? In the Hive 0.12 to Hive >> 0.13 upgrade script it calls hive-txn-schema-0.13.0.mysql.sql, which wil= l >> create these tables. If you're upgrading from 13 to 14 there shouldn't = be >> a need to create the tables. >> >> Alan. >> >> yogendra reddy >> November 30, 2014 at 21:19 >> Hi All, >> >> I'm caught up with the error pasted below after trying to use hive >> transaction feature.I have followed this wiki to set required >> configuration. >> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveT= ransactions-Configuration >> >> >> set hive.support.concurrency=3Dtrue; >> set hive.exec.dynamic.partition.mode=3Dnonstrict; >> set hive.txn.manager=3Dorg.apache.hadoop.hive.ql.lockmgr.DbTxnManager; >> set hive.compactor.initiator.on=3Dtrue; >> set hive.compactor.worker.threads=3D1; >> >> After setting the above I just ran this command "describe formatted >> acidtest;" >> any pointers as to why this occurs? >> >> FAILED: Error in acquiring locks: Error communicating with the metastore >> 14/12/01 05:02:05 ERROR ql.Driver: FAILED: Error in acquiring locks: >> Error communicating with the metastore >> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating wit= h >> the metastore >> at >> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:= 101) >> at >> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager= .java:202) >> at >> org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:939) >> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1166= ) >> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004) >> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994) >> at >> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247) >> at >> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199) >> at >> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410) >> at >> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783) >> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677) >> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav= a:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor= Impl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at org.apache.hadoop.util.RunJar.run(RunJar.java:221) >> at org.apache.hadoop.util.RunJar.main(RunJar.java:136) >> Caused by: org.apache.thrift.TApplicationException: Internal error >> processing lock >> at >> org.apache.thrift.TApplicationException.read(TApplicationException.java:= 108) >> at >> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71) >> at >> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_loc= k(ThriftHiveMetastore.java:3781) >> at >> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(Thr= iftHiveMetastore.java:3768) >> at >> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreC= lient.java:1736) >> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor= Impl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(Retrying= MetaStoreClient.java:90) >> at com.sun.proxy.$Proxy9.lock(Unknown Source) >> at >> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:= 80) >> ... 17 more >> >> p.s : Before setting the transaction specific properties I was able to >> run hive queries successfully >> >> Thanks, >> Yogendra >> >> >> -- >> Sent with Postbox >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidentia= l, >> privileged and exempt from disclosure under applicable law. If the reade= r >> of this message is not the intended recipient, you are hereby notified t= hat >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediat= ely >> and delete it from your system. Thank You. >> > --001a11c1fca2f76f5605095bec68 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks=C2=A0Vikas Srivastava, I do have lot of space available in my meta= store db


Thanks,
Yogendra

On Thu, Dec 4, 2014 at 9:12 AM, hado= op hive <hadoophive@gmail.com> wrote:

Hi yogendra,

You need to check if space is available on you backend metad= ata db server like mysql or oracle server.

It might be possible the your queries are getting stuck beco= z there is no enough space for mysql logging.

Thanks
Vikas Srivastava

On Dec 4, 2014 7:59 AM, "Yogendra reddy&quo= t; <yogendra.= 60@gmail.com> wrote:
Thanks Alan, I'm able to see TXNS tabl= e and yes I have upgraded from 0.13 to 0.14.

any idea as to wht els= e might cause this issue?

Thanks,
Yogendra
From: Al= an Gates
Sent: =E2=80=8E12/=E2=80=8E3/=E2=80=8E2014 9:51 AM
To: user@hiv= e.apache.org
Subject: Re: hive transactions

Usually this means the=20 metastore database was not properly upgraded.=C2=A0 You can check this by= =20 logging into your RDBMS and listing all the tables.=C2=A0 If you see a tabl= e=20 named TXNS then the proper tables exist.=C2=A0

Did you install Hive 0.14 fresh or upgrade from a previous version?=C2=A0 I= f=20 you upgraded did you run the upgrade scripts?=C2=A0 In the Hive 0.12 to Hiv= e=20 0.13 upgrade script it calls hive-txn-schema-0.13.0.mysql.sql, which=20 will create these tables.=C2=A0 If you're upgrading from 13 to 14 there= =20 shouldn't be a need to create the tables.

Alan.

=20 November 30, 201= 4 at 21:19
=
Hi All,

I'm caught up with the error pasted below after trying to use hive=20 transaction feature.I have followed this wiki to set required=20 configuration. https:/= /cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransaction= s-Configuration=C2=A0

set hive.support.concurrency=3Dtrue;
set=20 hive.exec.dynamic.partition.mode=3Dnonstrict;
set=20 hive.txn.manager=3Dorg.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
set hive.compactor.initiator.on=3Dtrue;
set=20 hive.compactor.worker.threads=3D1;

After=20 setting the above I just ran this command "describe formatted acidtest= ;"
any pointers as to why this occurs?

FAILED: Error=20 in acquiring locks: Error communicating with the metastore
14/12/= 01 05:02:05 ERROR ql.Driver: FAILED: Error in acquiring locks: Error=20 communicating with the metastore
org.apache.hadoop.hive.ql.lockmg= r.LockException: Error communicating with the metastore
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 at=20 org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:101= )
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.ja= va:202)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:939)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.hadoop.hive.ql.Driver.runInternal(Drive= r.java:1166)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1= 004)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.hadoop.hive.ql.Driver.run(Driver.java:9= 94)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
<= div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver= .java:677)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.hadoop.hive.cli.CliDriver.main(CliDrive= r.java:616)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ= e Method)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:5= 7)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp= l.java:43)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at java.lang.reflect.Method.invoke(Method.java:606)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
<= div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
=
Caused by: org.apache.thrift.TApplicationException: Internal error processing=20 lock
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.thrift.TApplicationException.read(TApplicationException.java:108= )
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
<= div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(T= hriftHiveMetastore.java:3781)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(Thrift= HiveMetastore.java:3768)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClie= nt.java:1736)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at sun.reflect.GeneratedMethodAccessor3.invoke(Unknow= n Source)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp= l.java:43)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at java.lang.reflect.Method.invoke(Method.java:606)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at=20 org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMet= aStoreClient.java:90)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at com.sun.proxy.$Proxy9.lock(Unknown Source)
=C2=A0 =C2=A0 =C2=A0 =C2=A0=20 at=20 org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:80)=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ... 17 more

p.s : Before set= ting the=20 transaction specific properties I was able to run hive queries=20 successfully=C2=A0

Thanks,
Yogendra

--
Sent with Postbox

CONFIDENTIALITY NOTICE
NOTICE: This message is = intended for the use of the individual or entity to which it is addressed a= nd may contain information that is confidential, privileged and exempt from= disclosure under applicable law. If the reader of this message is not the = intended recipient, you are hereby notified that any printing, copying, dis= semination, distribution, disclosure or forwarding of this communication is= strictly prohibited. If you have received this communication in error, ple= ase contact the sender immediately and delete it from your system. Thank Yo= u.

--001a11c1fca2f76f5605095bec68-- --001a11c1fca2f76f5b05095bec69 Content-Type: IMAGE/JPEG; name="compose-unknown-contact.jpg" Content-Disposition: inline; filename="compose-unknown-contact.jpg" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: 90ed4845aaa52063_0.1 /9j/4AAQSkZJRgABAQEARwBHAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQECAQEB AQEBAgICAgICAgICAgICAgICAgICAgICAgICAgICAgL/2wBDAQEBAQEBAQICAgICAgICAgICAgIC AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgL/wAARCAAZABkDAREA AhEBAxEB/8QAGAAAAwEBAAAAAAAAAAAAAAAABgcICQr/xAA0EAABAwMCAgUKBwAAAAAAAAACAQME BQYRABITIQcUMUF2CBUXIjI2N0JRtVRWkZOV0dL/xAAYAQEAAwEAAAAAAAAAAAAAAAADAAEEAv/E ACQRAAICAAQGAwAAAAAAAAAAAAABAhEDMrHREyExM0FxgfDx/9oADAMBAAIRAxEAPwDuEt+gW/UL et6oVC3rfqNQqFv0OfPn1GhUqfOmzZtKZlS5UqZMaNwzNwiJVIl7eXLCaZIGwBl3TY8epPx2+jy2 ZNPjvkwc9uhW8j7nCPhvOsQliYIeS7cvCpp8o50qwrC4v3lsNSDbdmTEhvs2tahxpfV3WnmbbozJ Ew/gwdadbYExVRXKEKoSdvJcaOSqxE7/AAiX0gXx+a69/JSf9alIlste0VzaNpeFrcT9KKymotyi aZ0KRCnzacoE7Kjzn4gi2KqUh3jqDHDHv4mRUfruTWlMzlVUKIVNp9GguEJnAh0+IZjyAiisgyRD nu5azS8miKqjOTVkKqS/psG37fo1Fbabeg25b8eZPeFJBBJSjMG5HjMeyihnaauZwe4OGiju13GA cpOwBeN+U8/IkGbsiS8b7ryogmbzhbyc9REROfZhERO5ETShjPtvpGqTUyLErytS4siSwx5x2tRH 4hPOI0DkjZtaJtFxuVEbIUUiyeNujlBUJGbJN6nM/Cyf2Hf60YgjvKA+NPSP4gT7axpcPtr51YWJ nYn9dnAQWl722p4ot37yzqnlfp6FrqbwawG8/9k= --001a11c1fca2f76f5b05095bec69--