hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <billgra...@gmail.com>
Subject Re: Hive query web service
Date Wed, 21 Oct 2009 16:32:06 GMT
There is no J2EE web server or SOAP web service in this equation. The Hive
JDBC client connects to the Hive Server, which can be started with a script
like so run from your $HIVE_HOME/build/dist directory:

export HADOOP_HOME=/path/to/hadoop
HIVE_PORT=10000 ./bin/hive --service hiveserver

No war files, or WEB-INF/ directories at all in this case.

On Wed, Oct 21, 2009 at 5:24 AM, Arijit Mukherjee <arijit72@gmail.com>wrote:

> If I understood the concept of "standalone" and "embedded" properly,
> then a Web Service which connects to the Hive/Thrift server via JDBC
> (jdbc:hive://host:port...) is actually a standalone client. The
> difference from a Java standalone client is that - in this case, the
> whole thing is packaged as a Web Service and deployed on a web server
> such as JBoss/GlassFish - and the connection is initiated only after a
> SOAP request is received from the web service client. If that is the
> case, then the Web Service should not require the conf files, or jpox
> libraries - is it not? Or did I misunderstand the concept?
>
> Arijit
>
> 2009/10/21 Arijit Mukherjee <arijit72@gmail.com>:
> > Update: I did a clean/build/deploy - the config files are within the
> > Web Service WEB-INF/classes folder, and the libraries (including the
> > jpox ones) are inside WEB-INF/lib - which are standard for any web
> > application. But the config related exception is still there:-((
> >
> > Arijit
> >
> > 2009/10/21 Arijit Mukherjee <arijit72@gmail.com>:
> >> Thanx Bill. I copied the jpox jars from the 0.3.0 distribution and
> >> added them to the web service archive, and they are in the classpath,
> >> but the config related exception is still there. Let me do a clean
> >> build/deploy and I'll get back again.
> >>
> >> Arijit
> >>
> >> 2009/10/20 Bill Graham <billgraham@gmail.com>:
> >>> The Hive JDBC client can run in two different modes: standalone and
> >>> embedded.
> >>>
> >>> Standalone mode is where the client connects to a separate standalone
> >>> HiveServer by specifying the host:port of the server in the jdbc URL
> like
> >>> this: jdbc:hive://localhost:10000/default. In this case the hive
> configs are
> >>> not needed by the client, since the client is making thrift requests to
> the
> >>> server which has the Hive configs. the Hive Server knows how to resolve
> the
> >>> metastore.
> >>>
> >>> Embedded mode is where the JDBC client "connects to itself" so to speak
> >>> using a JDBC url like this: jdbc:hive://. It's as if the client is
> running
> >>> an embedded server that only it communicates with. In this case the
> client
> >>> needs the Hive configs since it needs to resolve the metastore, amongst
> >>> other things. The metastore dependency in this case is what will cause
> you
> >>> to see jpox errors appear if those jars aren't found.
> >>>
> >>> HTH,
> >>> Bill
> >>>
> >>> On Tue, Oct 20, 2009 at 4:14 AM, Arijit Mukherjee <arijit72@gmail.com>
> >>> wrote:
> >>>>
> >>>> BTW - the service is working though, in spite of those exceptions. I'm
> >>>> able to run queries and get results.
> >>>>
> >>>> Arijit
> >>>>
> >>>> 2009/10/20 Arijit Mukherjee <arijit72@gmail.com>:
> >>>> > I created a hive-site.xml using the outline given in the Hive Web
> >>>> > Interface tutorial - now that file is in the classpath of the Web
> >>>> > Service - and the service can find the file. But, now there's
> another
> >>>> > exception -
> >>>> >
> >>>> > 2009-10-20 14:27:30,914 DEBUG [httpSSLWorkerThread-14854-0]
> >>>> > HiveQueryService - connecting to Hive using URL:
> >>>> > jdbc:hive://localhost:10000/default
> >>>> > 2009-10-20 14:27:30,969 DEBUG [httpSSLWorkerThread-14854-0]
> >>>> > Configuration - java.io.IOException: config()
> >>>> >        at
> >>>> > org.apache.hadoop.conf.Configuration.<init>(Configuration.java:176)
> >>>> >        at
> >>>> > org.apache.hadoop.conf.Configuration.<init>(Configuration.java:164)
> >>>> >        at
> org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:287)
> >>>> >        at
> >>>> >
> org.apache.hadoop.hive.jdbc.HiveConnection.<init>(HiveConnection.java:63)
> >>>> >        at
> >>>> > org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:109)
> >>>> >        at
> java.sql.DriverManager.getConnection(DriverManager.java:582)
> >>>> >        at
> java.sql.DriverManager.getConnection(DriverManager.java:185)
> >>>> >        at
> >>>> >
> com.ctva.poc.hive.service.HiveQueryService.getConnection(HiveQueryService.java:134)
> >>>> >        at
> >>>> >
> com.ctva.poc.hive.service.HiveQueryService.connectDB(HiveQueryService.java:43)
> >>>> >
> >>>> > Apparently, something goes wrong during the config routine. Do
I
> need
> >>>> > something more within the service?
> >>>> >
> >>>> > Regards
> >>>> > Arijit
> >>>> >
> >>>> > 2009/10/20 Arijit Mukherjee <arijit72@gmail.com>:
> >>>> >> Hi
> >>>> >>
> >>>> >> I'm trying to create a Web Service which will access Hive (0.4.0
> >>>> >> release) using JDBC. I used to sample JDBC code from the wiki
> >>>> >>
> >>>> >> (
> http://wiki.apache.org/hadoop/Hive/HiveClient#head-fd2d8ae9e17fdc3d9b7048d088b2c23a53a6857d
> ),
> >>>> >> but when I'm trying to connect the the DB using the DriverManager,
> >>>> >> there's an exception which seems to relate to hive-site.xml
> (HiveConf
> >>>> >> - hive-site.xml not found.). But I could not find any hive-site.xml
> in
> >>>> >> $HIVE_HOME/conf - there's only hive-default.xml. The wiki page
also
> >>>> >> speaks about couple of jpox JAR files, which aren't in the
lib
> folder
> >>>> >> either.
> >>>> >>
> >>>> >> Am I missing something here?
> >>>> >>
> >>>> >> Regards
> >>>> >> Arijit
> >>>> >>
> >>>> >> --
> >>>> >> "And when the night is cloudy,
> >>>> >> There is still a light that shines on me,
> >>>> >> Shine on until tomorrow, let it be."
> >>>> >>
> >>>> >
> >>>> >
> >>>> >
> >>>> > --
> >>>> > "And when the night is cloudy,
> >>>> > There is still a light that shines on me,
> >>>> > Shine on until tomorrow, let it be."
> >>>> >
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> "And when the night is cloudy,
> >>>> There is still a light that shines on me,
> >>>> Shine on until tomorrow, let it be."
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> "And when the night is cloudy,
> >> There is still a light that shines on me,
> >> Shine on until tomorrow, let it be."
> >>
> >
> >
> >
> > --
> > "And when the night is cloudy,
> > There is still a light that shines on me,
> > Shine on until tomorrow, let it be."
> >
>
>
>
> --
> "And when the night is cloudy,
> There is still a light that shines on me,
> Shine on until tomorrow, let it be."
>

Mime
View raw message