Mailing-List: contact hive-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hive-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of arijit72@gmail.com designates
 209.85.216.198 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=TXAbug5xWV3IiG7rJjQF3B/y9bR5P35fojZn7TfSxS9iWqG9l/MWhjvwmmtbEGhgIm
         Nt3h1ywwa9Oj4NBBKtW6TcFnfTQY2dZHqNFNwT6Dqdz2OaWwjNQWSA+RvDc0QLAdotJX
         hgqy55DhC2mYaz5LTjUE1uDLv50H2ycEER8Bc=
MIME-Version: 1.0
In-Reply-To: <449b48760910210931i588ef77cgfeabe3b1d8ac23fc@mail.gmail.com>
References: <4cfc4d490910200154i3b58415bsfdaf95af406fcb07@mail.gmail.com>
	 <4cfc4d490910200210p37f0df79lbfd3738ff6ed7165@mail.gmail.com>
	 <4cfc4d490910200414n5524e805s43eb849708e4cd25@mail.gmail.com>
	 <449b48760910200927w5cf423e6qe49d03a54b721d55@mail.gmail.com>
	 <4cfc4d490910202135j406ff959n8a7ce06192024a2d@mail.gmail.com>
	 <4cfc4d490910202218i6c07e2eew437489d93419f239@mail.gmail.com>
	 <4cfc4d490910210524q71c5d182r9b6e4ff04942a1bc@mail.gmail.com>
	 <449b48760910210931i588ef77cgfeabe3b1d8ac23fc@mail.gmail.com>
Date: Thu, 22 Oct 2009 09:39:40 +0530
Message-ID: <4cfc4d490910212109t141347a7s8113e78b727edae@mail.gmail.com>
Subject: Re: Hive query web service
From: Arijit Mukherjee <arijit72@gmail.com>
To: billgraham@gmail.com, hive-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Yes - I understood that. But this is what I am doing - implementing a
Web Service which will connect to the Hive/Thrift server via JDBC and
execute queries. For that, I used some statements to connect as shown
in the sample JDBC application.

Arijit

2009/10/21 Bill Graham <billgraham@gmail.com>:
> There is no J2EE web server or SOAP web service in this equation. The Hiv=
e
> JDBC client connects to the Hive Server, which can be started with a scri=
pt
> like so run from your $HIVE_HOME/build/dist directory:
>
> export HADOOP_HOME=3D/path/to/hadoop
> HIVE_PORT=3D10000 ./bin/hive --service hiveserver
>
> No war files, or WEB-INF/ directories at all in this case.
>
>
> On Wed, Oct 21, 2009 at 5:24 AM, Arijit Mukherjee <arijit72@gmail.com>
> wrote:
>>
>> If I understood the concept of "standalone" and "embedded" properly,
>> then a Web Service which connects to the Hive/Thrift server via JDBC
>> (jdbc:hive://host:port...) is actually a standalone client. The
>> difference from a Java standalone client is that - in this case, the
>> whole thing is packaged as a Web Service and deployed on a web server
>> such as JBoss/GlassFish - and the connection is initiated only after a
>> SOAP request is received from the web service client. If that is the
>> case, then the Web Service should not require the conf files, or jpox
>> libraries - is it not? Or did I misunderstand the concept?
>>
>> Arijit
>>
>> 2009/10/21 Arijit Mukherjee <arijit72@gmail.com>:
>> > Update: I did a clean/build/deploy - the config files are within the
>> > Web Service WEB-INF/classes folder, and the libraries (including the
>> > jpox ones) are inside WEB-INF/lib - which are standard for any web
>> > application. But the config related exception is still there:-((
>> >
>> > Arijit
>> >
>> > 2009/10/21 Arijit Mukherjee <arijit72@gmail.com>:
>> >> Thanx Bill. I copied the jpox jars from the 0.3.0 distribution and
>> >> added them to the web service archive, and they are in the classpath,
>> >> but the config related exception is still there. Let me do a clean
>> >> build/deploy and I'll get back again.
>> >>
>> >> Arijit
>> >>
>> >> 2009/10/20 Bill Graham <billgraham@gmail.com>:
>> >>> The Hive JDBC client can run in two different modes: standalone and
>> >>> embedded.
>> >>>
>> >>> Standalone mode is where the client connects to a separate standalon=
e
>> >>> HiveServer by specifying the host:port of the server in the jdbc URL
>> >>> like
>> >>> this: jdbc:hive://localhost:10000/default. In this case the hive
>> >>> configs are
>> >>> not needed by the client, since the client is making thrift requests
>> >>> to the
>> >>> server which has the Hive configs. the Hive Server knows how to
>> >>> resolve the
>> >>> metastore.
>> >>>
>> >>> Embedded mode is where the JDBC client "connects to itself" so to
>> >>> speak
>> >>> using a JDBC url like this: jdbc:hive://. It's as if the client is
>> >>> running
>> >>> an embedded server that only it communicates with. In this case the
>> >>> client
>> >>> needs the Hive configs since it needs to resolve the metastore,
>> >>> amongst
>> >>> other things. The metastore dependency in this case is what will cau=
se
>> >>> you
>> >>> to see jpox errors appear if those jars aren't found.
>> >>>
>> >>> HTH,
>> >>> Bill
>> >>>
>> >>> On Tue, Oct 20, 2009 at 4:14 AM, Arijit Mukherjee <arijit72@gmail.co=
m>
>> >>> wrote:
>> >>>>
>> >>>> BTW - the service is working though, in spite of those exceptions.
>> >>>> I'm
>> >>>> able to run queries and get results.
>> >>>>
>> >>>> Arijit
>> >>>>
>> >>>> 2009/10/20 Arijit Mukherjee <arijit72@gmail.com>:
>> >>>> > I created a hive-site.xml using the outline given in the Hive Web
>> >>>> > Interface tutorial - now that file is in the classpath of the Web
>> >>>> > Service - and the service can find the file. But, now there's
>> >>>> > another
>> >>>> > exception -
>> >>>> >
>> >>>> > 2009-10-20 14:27:30,914 DEBUG [httpSSLWorkerThread-14854-0]
>> >>>> > HiveQueryService - connecting to Hive using URL:
>> >>>> > jdbc:hive://localhost:10000/default
>> >>>> > 2009-10-20 14:27:30,969 DEBUG [httpSSLWorkerThread-14854-0]
>> >>>> > Configuration - java.io.IOException: config()
>> >>>> > =A0 =A0 =A0 =A0at
>> >>>> > org.apache.hadoop.conf.Configuration.<init>(Configuration.java:17=
6)
>> >>>> > =A0 =A0 =A0 =A0at
>> >>>> > org.apache.hadoop.conf.Configuration.<init>(Configuration.java:16=
4)
>> >>>> > =A0 =A0 =A0 =A0at
>> >>>> > org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:287)
>> >>>> > =A0 =A0 =A0 =A0at
>> >>>> >
>> >>>> > org.apache.hadoop.hive.jdbc.HiveConnection.<init>(HiveConnection.=
java:63)
>> >>>> > =A0 =A0 =A0 =A0at
>> >>>> > org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:10=
9)
>> >>>> > =A0 =A0 =A0 =A0at
>> >>>> > java.sql.DriverManager.getConnection(DriverManager.java:582)
>> >>>> > =A0 =A0 =A0 =A0at
>> >>>> > java.sql.DriverManager.getConnection(DriverManager.java:185)
>> >>>> > =A0 =A0 =A0 =A0at
>> >>>> >
>> >>>> > com.ctva.poc.hive.service.HiveQueryService.getConnection(HiveQuer=
yService.java:134)
>> >>>> > =A0 =A0 =A0 =A0at
>> >>>> >
>> >>>> > com.ctva.poc.hive.service.HiveQueryService.connectDB(HiveQuerySer=
vice.java:43)
>> >>>> >
>> >>>> > Apparently, something goes wrong during the config routine. Do I
>> >>>> > need
>> >>>> > something more within the service?
>> >>>> >
>> >>>> > Regards
>> >>>> > Arijit
>> >>>> >
>> >>>> > 2009/10/20 Arijit Mukherjee <arijit72@gmail.com>:
>> >>>> >> Hi
>> >>>> >>
>> >>>> >> I'm trying to create a Web Service which will access Hive (0.4.0
>> >>>> >> release) using JDBC. I used to sample JDBC code from the wiki
>> >>>> >>
>> >>>> >>
>> >>>> >> (http://wiki.apache.org/hadoop/Hive/HiveClient#head-fd2d8ae9e17f=
dc3d9b7048d088b2c23a53a6857d),
>> >>>> >> but when I'm trying to connect the the DB using the DriverManage=
r,
>> >>>> >> there's an exception which seems to relate to hive-site.xml
>> >>>> >> (HiveConf
>> >>>> >> - hive-site.xml not found.). But I could not find any
>> >>>> >> hive-site.xml in
>> >>>> >> $HIVE_HOME/conf - there's only hive-default.xml. The wiki page
>> >>>> >> also
>> >>>> >> speaks about couple of jpox JAR files, which aren't in the lib
>> >>>> >> folder
>> >>>> >> either.
>> >>>> >>
>> >>>> >> Am I missing something here?
>> >>>> >>
>> >>>> >> Regards
>> >>>> >> Arijit
>> >>>> >>
>> >>>> >> --
>> >>>> >> "And when the night is cloudy,
>> >>>> >> There is still a light that shines on me,
>> >>>> >> Shine on until tomorrow, let it be."
>> >>>> >>
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> > "And when the night is cloudy,
>> >>>> > There is still a light that shines on me,
>> >>>> > Shine on until tomorrow, let it be."
>> >>>> >
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> "And when the night is cloudy,
>> >>>> There is still a light that shines on me,
>> >>>> Shine on until tomorrow, let it be."
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> "And when the night is cloudy,
>> >> There is still a light that shines on me,
>> >> Shine on until tomorrow, let it be."
>> >>
>> >
>> >
>> >
>> > --
>> > "And when the night is cloudy,
>> > There is still a light that shines on me,
>> > Shine on until tomorrow, let it be."
>> >
>>
>>
>>
>> --
>> "And when the night is cloudy,
>> There is still a light that shines on me,
>> Shine on until tomorrow, let it be."
>
>


--=20
"And when the night is cloudy,
There is still a light that shines on me,
Shine on until tomorrow, let it be."