hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuart Smith <>
Subject Re: Using jdbc in embedded mode - Can't find warehouse directory [SOLVED]
Date Sat, 20 Nov 2010 00:57:27 GMT


  Just wanted to let people know I tracked this one down:

It looks like it was not picking up the *hadoop* core-site.xml configuration file. 

- So the variable was never set

- So the warehouse dir became file://[hive.metastore.warehouse.dir] instead of [hdfs location]/[hive.metastore.warehouse.dir]

- So it couldn't find any of the warehouse files.

- So the metastore queries would start to work, but the metastore couldn't find any of the
backing files on hdfs.

It was picking up the hive configuration, so I just plopped the property from
hdfs-site.xml into the hive configuration.

Should the jdbc wiki:

Be updated to include this information?

It could be useful to anyone trying to use an embedded server (vs the example given). I would
actually think this would apply to the standalone case as well, but I haven't tried it yet.

My particular use case is using the jdbc connector in a java servlet (specifically, a GWT
server-side RPC implementation).

As an aside: is the hive jdbc connector thread-safe? 
Assuming I instantiate within the callback method?
(I would think having a class Connection member would not be thread safe?).

I'd be happy to help update the wiki & come up with an example, if that would help..

Take care,

--- On Thu, 11/18/10, Stuart Smith <> wrote:

> From: Stuart Smith <>
> Subject: Using jdbc in embedded mode - Can't find warehouse directory
> To:
> Date: Thursday, November 18, 2010, 7:46 PM
> Hello,
>   I'm trying to connect to hive using the JDBC driver
> in embedded mode. I can load the driver successfully &
> connect to it via:
> hiveConnection = DriverManager.getConnection(
> "jdbc:hive://", "", "" )
> But when I query a table that I know exists - I can query
> it via a hive command line running on the same machine - I
> get a "table does not exist" error. When I go ahead and
> create the table in my java program, and then query it, I
> get: 
> ERROR: hive.log File
> file:/user/hive/warehouse/[table_name]
>         at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(
> ...
> So it looks like it's trying to use the local filesystem
> for the warehouse dir. I tried setting the warehouse dir
> variable in the hive-default.xml file to:
> hdfs://user/hive/warehouse/
> But I get the same errors.
> Any idea what's happening? 
> Am I confused on what an embedded hive server can do - I
> was under the impression that the cli used an embedded hive
> server, and could connect to my hdfs store, but... it would
> seem my java program can't this.
> I guess my next stop is going through the hive cli source
> code ?
> Take care,
>   -stu


View raw message