hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Sichi <jsi...@fb.com>
Subject Re: Hive/Hbase Integration Error
Date Thu, 06 Jan 2011 16:34:38 GMT
Here is what you need to do:

1) Use svn to check out the source for Hive 0.6

2) In your checkout, replace the HBase 0.20.3 jars with the ones from 0.20.6

3) Build Hive 0.6 from source

4) Use your new Hive build

JVS

On Jan 6, 2011, at 2:34 AM, Adarsh Sharma wrote:

> Dear all,
> 
> I am sorry I am posting this message again but I can't able to locate the root cause
after googled a lot.
> 
> I am trying Hive/Hbase Integration from the past 2 days. I am facing the below issue
while creating external table in Hive.
> 
> I am using hadoop-0.20.2, hbase-0.20.6, hive-0.6.0 ( Mysql as metstore ) and java-1.6.0_20.
Hbase-0.20.3 is also checked.
> 
> Problem arises when I issue the below command :
> 
> hive> CREATE TABLE hive_hbasetable_k(key int, value string)
>     > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>     > WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
>     > TBLPROPERTIES ("hbase.table.name" = "hivehbasek");
> 
> 
> FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
>         at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
>         at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
>         at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:64)
>         at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:159)
>         at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:275)
>         at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:394)
>         at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2126)
>         at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:166)
>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
>         at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:633)
>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:506)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:384)
>         at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:302)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
> 
> 
> It seems my HMaster is not Running but I checked from IP:60010 that it is running and
I am able to create,insert tables in Hbase Properly.
> 
> Below is the contents of my hive.log :
> 
>   2011-01-05 15:19:36,783 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle
"org.eclipse.jdt.core" requires "org.eclipse.core.resources" but it cannot be resolved.
>  2011-01-05 15:19:36,783 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle
"org.eclipse.jdt.core" requires "org.eclipse.core.resources" but it cannot be resolved.
>  2011-01-05 15:19:36,785 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle
"org.eclipse.jdt.core" requires "org.eclipse.core.runtime" but it cannot be resolved.
>  2011-01-05 15:19:36,785 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle
"org.eclipse.jdt.core" requires "org.eclipse.core.runtime" but it cannot be resolved.
>  2011-01-05 15:19:36,786 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle
"org.eclipse.jdt.core" requires "org.eclipse.text" but it cannot be resolved.
>  2011-01-05 15:19:36,786 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle
"org.eclipse.jdt.core" requires "org.eclipse.text" but it cannot be resolved.
>  2011-01-05 15:20:12,185 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(967)) - Exception
closing session 0x0 to sun.nio.ch.SelectionKeyImpl@561279c8
>  java.net.ConnectException: Connection refused
>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:933)
>  2011-01-05 15:20:12,188 WARN  zookeeper.ClientCnxn (ClientCnxn.java:cleanup(1001)) -
Ignoring exception during shutdown input
>  java.nio.channels.ClosedChannelException
>        at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
>        at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
>        at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
>        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
>  2011-01-05 15:20:12,188 WARN  zookeeper.ClientCnxn (ClientCnxn.java:cleanup(1006)) -
Ignoring exception during shutdown output
>  java.nio.channels.ClosedChannelException
>        at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:649)
>        at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
>        at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
>        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
>  2011-01-05 15:20:12,621 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(967)) - Exception
closing session 0x0 to sun.nio.ch.SelectionKeyImpl@799dbc3b
>  
>   Please help me, as i am not able to solve this problem.
>  
>  Also I want to add one more thing that my hadoop Cluster is of 9 nodes and 8 nodes act
as Datanodes,Tasktrackers and Regionservers.
>  
>  Among these nodes is set zookeeper.quorum.property to have 5 Datanodes. I don't know
the number of servers needed for Zookeeper in fully distributed mode.
>  
>  
>  Best Regards
> 
>  Adarsh Sharma
> 
> 
> 


Mime
View raw message