hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adarsh Sharma <adarsh.sha...@orkash.com>
Subject Re: Hive/Hbase Integration Error
Date Tue, 11 Jan 2011 04:18:25 GMT
Jean-Daniel Cryans wrote:
> Sorry it that wasn't obvious, but you need to run hive using this command:
>
> bin/hive -hiveconf hive.root.logger=INFO,console
>
> AND in the same shell create the table in order to have more
> information. What we're trying to get looks like this:
> http://pastebin.com/gvTXDxtE
>
> Specifically, this line is very important (it should have different
> values in your case):
>
> INFO zookeeper.ClientCnxn: Priming connection to
> java.nio.channels.SocketChannel[connected local=/10.10.20.42:53187
> remote=sv4borg13/10.10.21.13:2181]
>
> Regarding your second log paste, it could be harmless if the process
> on that other machine just took more time to boot, also it happened 20
> minutes before your test. Do verify that hbase works before trying to
> create a table.
>
> J-D
>
> On Sun, Jan 9, 2011 at 10:37 PM, Adarsh Sharma <adarsh.sharma@orkash.com> wrote:
>   
>> Jean-Daniel Cryans wrote:
>>
>> You also need to create the table in order to see the relevant debug
>> information, it won't create it until it needs it.
>>
>> Sir
>> Check the output :
>>
>> hive> CREATE TABLE hive_hbasetable_k(key int, value string)
>>     > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>>     > WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
>>     > TBLPROPERTIES ("hbase.table.name" = "hivehbasek");
>> FAILED: Error in metadata:
>> MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
>>         at
>> org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
>>         at
>> org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:64)
>>         at
>> org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:159)
>>         at
>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:275)
>>         at
>> org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:394)
>>         at
>> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2126)
>>         at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:166)
>>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
>>         at
>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
>>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:633)
>>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:506)
>>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:384)
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
>>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:302)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>> )
>> FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.DDLTask
>>
>>
>> And the head of Hive.log says :
>>
>> 2011-01-10 11:57:59,467 ERROR DataNucleus.Plugin
>> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
>> "org.eclipse.core.resources" but it cannot be resolved.
>> 2011-01-10 11:57:59,470 ERROR DataNucleus.Plugin
>> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
>> "org.eclipse.core.runtime" but it cannot be resolved.
>> 2011-01-10 11:57:59,470 ERROR DataNucleus.Plugin
>> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
>> "org.eclipse.core.runtime" but it cannot be resolved.
>> 2011-01-10 11:57:59,471 ERROR DataNucleus.Plugin
>> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
>> "org.eclipse.text" but it cannot be resolved.
>> 2011-01-10 11:57:59,471 ERROR DataNucleus.Plugin
>> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
>> "org.eclipse.text" but it cannot be resolved.
>> 2011-01-10 11:58:38,210 WARN  zookeeper.ClientCnxn
>> (ClientCnxn.java:run(967)) - Exception closing session 0x0 to
>> sun.nio.ch.SelectionKeyImpl@6070c38c
>> java.io.IOException: TIMED OUT
>>         at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:906)
>> 2011-01-10 11:58:38,215 WARN  zookeeper.ClientCnxn
>> (ClientCnxn.java:cleanup(1006)) - Ignoring exception during shutdown output
>> java.net.SocketException: Transport endpoint is not connected
>>         at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
>>         at
>> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
>>         at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
>>         at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
>>         at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
>> 2011-01-10 11:58:39,019 WARN  zookeeper.ClientCnxn
>> (ClientCnxn.java:run(967)) - Exception closing session 0x0 to
>> sun.nio.ch.SelectionKeyImpl@34b6a6d6
>> java.net.ConnectException: Connection refused
>>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>         at
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>         at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:933)
>>
>> 1,1           Top
>>
>> And I am researching on Zookeeper errors in Hbase logs that says :
>>
>> Mon Jan 10 11:35:01 IST 2011 Starting zookeeper on s1-ser
>> ulimit -n 1024
>> 2011-01-10 11:35:01,703 INFO
>> org.apache.zookeeper.server.quorum.QuorumPeerConfig: Defaulting to majority
>> quorums
>> 2011-01-10 11:35:01,759 DEBUG org.apache.hadoop.hbase.zookeeper.HQuorumPeer:
>> preRegister called. Server=com.sun.jmx.mbeanserver.JmxMBeanServer@2d342ba4,
>> name=log4j:logger=org.apache.hadoop.hbase.zookeeper.HQuorumPeer
>> 2011-01-10 11:35:01,759 DEBUG org.apache.hadoop.hbase: preRegister called.
>> Server=com.sun.jmx.mbeanserver.JmxMBeanServer@2d342ba4,
>> name=log4j:logger=org.apache.hadoop.hbase
>> 2011-01-10 11:35:01,759 INFO
>> org.apache.zookeeper.server.quorum.QuorumPeerMain: Starting quorum peer
>> 2011-01-10 11:35:01,807 INFO
>> org.apache.zookeeper.server.quorum.QuorumCnxManager: My election bind port:
>> 3888
>> 2011-01-10 11:35:01,833 INFO org.apache.zookeeper.server.quorum.QuorumPeer:
>> LOOKING
>> 2011-01-10 11:35:01,837 INFO
>> org.apache.zookeeper.server.quorum.FastLeaderElection: New election:
>> 12884901931
>> 2011-01-10 11:35:01,842 INFO
>> org.apache.zookeeper.server.quorum.FastLeaderElection: Notification: 0,
>> 12884901931, 1, 0, LOOKING, LOOKING, 0
>> 2011-01-10 11:35:01,842 INFO
>> org.apache.zookeeper.server.quorum.FastLeaderElection: Adding vote
>> 2011-01-10 11:35:01,844 WARN
>> org.apache.zookeeper.server.quorum.QuorumCnxManager: Cannot open channel to
>> 1 at election address /192.168.1.103:3888
>> java.net.ConnectException: Connection refused
>>         at sun.nio.ch.Net.connect(Native Method)
>>         at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
>>         at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
>>         at
>> org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:323)
>>         at
>> org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:302)
>>         at
>> org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:323)
>>         at
>> org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:296)
>>         at java.lang.Thread.run(Thread.java:619)
>> 2011-01-10 11:35:01,850 INFO org.apache.zookeeper.server.quorum
>>
>>
>> If you require any other information, Please let me know.
>>
>>
>> Best regards
>>
>> Adarsh
>>
>>     
Sir, I want a suggestion from your side.

I configured hadoop-0.20.2 on 9 servers.

192.168.0.173 ( Namenode, Jobtracker , HMaster, Hive Package holder )
192.168.0.174.......182  ( datanodes , tasktracker , HRegionservers )
I set 192.168.0.174........176 to *zookeeper.quorum.property . *I think 
this causes error on Zookeeper logs.

On the other hand, I have a separate cluster of 4 nodes.
1 server ( Namenode, Jobtracker , HMaster, Hive Package holder )
Remaining 3 Servers ( datanodes , tasktracker , HRegionservers ).
I also set these 3 servers IP in *zookeeper.quorum.property.


*I have a doubt regarding *zookeeper.quorum.property. *Is it correct or 
I need separate servers for it.



Thanks

Mime
View raw message