Return-Path: Delivered-To: apmail-hadoop-hive-user-archive@locus.apache.org Received: (qmail 76433 invoked from network); 10 Dec 2008 07:31:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 10 Dec 2008 07:31:20 -0000 Received: (qmail 23314 invoked by uid 500); 10 Dec 2008 07:31:33 -0000 Delivered-To: apmail-hadoop-hive-user-archive@hadoop.apache.org Received: (qmail 23289 invoked by uid 500); 10 Dec 2008 07:31:33 -0000 Mailing-List: contact hive-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hive-user@hadoop.apache.org Delivered-To: mailing list hive-user@hadoop.apache.org Received: (qmail 23278 invoked by uid 99); 10 Dec 2008 07:31:33 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Dec 2008 23:31:32 -0800 X-ASF-Spam-Status: No, hits=3.4 required=10.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.200.168] (HELO wf-out-1314.google.com) (209.85.200.168) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Dec 2008 07:31:17 +0000 Received: by wf-out-1314.google.com with SMTP id 24so37757wfg.2 for ; Tue, 09 Dec 2008 23:30:54 -0800 (PST) Received: by 10.142.253.21 with SMTP id a21mr94475wfi.174.1228894253706; Tue, 09 Dec 2008 23:30:53 -0800 (PST) Received: from ?192.168.1.102? (c-71-202-152-68.hsd1.ca.comcast.net [71.202.152.68]) by mx.google.com with ESMTPS id 24sm244047wff.37.2008.12.09.23.30.52 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 09 Dec 2008 23:30:53 -0800 (PST) Message-Id: <89C27756-620A-460C-9EAD-72007E6938CE@besquared.net> From: Josh Ferguson To: hive-user@hadoop.apache.org In-Reply-To: Content-Type: multipart/alternative; boundary=Apple-Mail-6--668520381 Mime-Version: 1.0 (Apple Message framework v929.2) Subject: Re: Metadata in Multiuser DB Date: Tue, 9 Dec 2008 23:31:06 -0800 References: X-Mailer: Apple Mail (2.929.2) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail-6--668520381 Content-Type: text/plain; charset=WINDOWS-1252; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable What does this usually mean? hive> SHOW TABLES; FAILED: Error in metadata: MetaException(message:Got exception: =20 javax.jdo.JDOFatalInternalException Error creating transactional =20 connection factory) FAILED: Execution Error, return code 1 from =20 org.apache.hadoop.hive.ql.exec.DDLTask Time taken: 2.228 seconds Josh On Dec 9, 2008, at 9:15 AM, Joydeep Sen Sarma wrote: > We use mysql as metadb server. > > Prasad can give a more detailed response when he=92s back =96 but here = =20 > are the relevant entries from our hive-default.xml: > > > javax.jdo.option.ConnectionURL > jdbc:mysql://xxx.yyy.facebook.com/hms_during_upgrade?=20 > createDatabaseIfNotExist=3Dtrue > > > > javax.jdo.option.ConnectionDriverName > com.mysql.jdbc.Driver > > > > javax.jdo.option.ConnectionUserName > root > > > > javax.jdo.option.ConnectionPassword > xxxxxx > > > > org.jpox.autoCreateSchema > false > > > > org.jpox.fixedDatastore > true > > > > hive.metastore.usefilestore > false > > > > hive.metastore.checkForDefaultDb > false > > > > hive.metastore.rawstore.impl > org.apache.hadoop.hive.metastore.ObjectStore > Name of the class that implements =20 > org.apache.hadoop.hive.metastore.rawstore interface. This class is =20 > used to store and retrieval of raw metadata objects such as table, dat > abase > > > > hive.metastore.local > true > > > > - autoCreateSchema will probably have to be true for first =20= > time deployment. > - Not sure usefilestore is required or not =96 this is just a = =20 > vestige from older code that=92s saying that we want to use db backed =20= > metastore. No harm putting it there > - Hive.metastore.local =96 this is important since we want to = =20 > connect from hive cli directly to metastore (hence =91local=92) = (instead =20 > of through a thrift server) > - checkForDefaultDb =96 not entirely sure about this =96 but =20= > this was having some performance impact for us. > > Hope this helps, > > Joydeep > > > > From: Bill Au [mailto:bill.w.au@gmail.com] > Sent: Tuesday, December 09, 2008 7:21 AM > To: hive-user@hadoop.apache.org > Subject: Re: Metadata in Multiuser DB > > I followed the instruction in the wiki for using derby in network =20 > server mode and was able to get hive running in multi-user mode. I =20= > would be interested in using mysql instead. Not sure if I will have =20= > time to try to get that working so instructions from someone who has =20= > already done so would be very useful. > > Bill > On Tue, Dec 9, 2008 at 9:46 AM, Edward Capriolo =20 > wrote: > You have two options: > > 1) You can start derby in network server mode rather than embedded. > http://wiki.apache.org/hadoop/HiveDerbyServerMode > 2) You can also follow the above instructions and replaced derby with > an jpox capable database. > > If you were going to use another database besides derby server mode > like mysql. It would be cool if you added another wiki with > instructions. I have never tried mysql as a metastore but it may have > better performance/be more tunable then derby. > --Apple-Mail-6--668520381 Content-Type: text/html; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable
What does this usually = mean?

hive> SHOW TABLES;
FAILED: = Error in metadata: MetaException(message:Got = exception: javax.jdo.JDOFatalInternalException Error creating = transactional connection factory)
FAILED: Execution Error, = return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Time = taken: 2.228 = seconds

Josh

On = Dec 9, 2008, at 9:15 AM, Joydeep Sen Sarma wrote:

We use mysql as metadb = server.
 
Prasad can = give a more detailed response when he=92s back =96 but here are the = relevant entries from our = hive-default.xml:
<property>
  = <name>javax.jdo.option.ConnectionURL</name>=
  = <value>jdbc:mysql://xxx.yyy.facebook.com/hms_during_upgrade?createDatab= aseIfNotExist=3Dtrue</value>
<property>
  = <name>javax.jdo.option.ConnectionDriverName</name>=
  = <value>com.mysql.jdbc.Driver</value>
<= div style=3D"margin-top: 0in; margin-right: 0in; margin-left: 0in; = margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New = Roman'; "><property>
  = <name>javax.jdo.option.ConnectionUserName</name>
  = <value>root</value>
<property>
  = <name>javax.jdo.option.ConnectionPassword</name>
  = <value>xxxxxx</value>
<property>
  = <name>org.jpox.autoCreateSchema</name>
  = <value>false</value>
<property>
  = <name>org.jpox.fixedDatastore</name>
<= div style=3D"margin-top: 0in; margin-right: 0in; margin-left: 0in; = margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New = Roman'; ">  = <value>true</value>
<property>
  = <name>hive.metastore.usefilestore</name>
  = <value>false</value>
<property>
  = <name>hive.metastore.checkForDefaultDb</name>
  = <value>false</value>
<property>
  = <name>hive.metastore.rawstore.impl</name>
  = <value>org.apache.hadoop.hive.metastore.ObjectStore</value>
  = <description>Name of the class that implements = org.apache.hadoop.hive.metastore.rawstore interface. This class is used = to store and retrieval of raw metadata objects such as table, = dat
abase</description>
<property>
  = <name>hive.metastore.local</name>
  = <value>true</value>
 
 <= /font> <= /font>Not sure = usefilestore is required or not =96 this is just a vestige from older = code that=92s saying that we want to use db backed metastore. No harm = putting it there
 <= /font> <= /font> 
Hope this = helps,
 
 
 
From: Bill Au [mailto:bill.w.au@gmail.com] 
Sent: Tuesday, December 09, 2008 = 7:21 AM
To: Subject: Re: Metadata in Multiuser = DB
You have two = options:

1) You can start derby in network server mode rather = than embedded.