hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From exception qin <exceptionq...@gmail.com>
Subject Re: blocked when creating HTable
Date Tue, 07 Dec 2010 08:19:36 GMT
Hi,

I did some investigation on this issue.There are several things I need to
make clear.

1, I'm using Cloudera's Distribution for Hadoop(0.20.0),HBase(0.20.6) and
Zookeeper(3.3.1)
2, The HBase didn't manage it's own instance of zookeeper.
3, I found my program can connected to zookeeper successfully and didn't
interact with Hbase yet. When I stopped the HMaster process. The program
still blocked. It will throw exceptions when I stopped zookeeper.
4, It looks like the configuration in hbase-default.xml isn't overwrite by
those in hbase-site.xml. I have done some simple test.

HBaseConfiguration config = new HBaseConfiguration();
System.out.println(config.get("hbase.zookeeper.quorum",""));
System.out.println(config.get("hbase.cluster.distributed",""));
System.out.println(config.get("hbase.rootdir",""));

 This will print the default value in hbase-default.xml. I overwrite
these manually in my program but it still doesn't work.

Can someone tell me what did the HBase do when constructing HTable after
connecting to zookeeper?

Thank.


On Tue, Dec 7, 2010 at 9:34 AM, exception qin <exceptionqin2@gmail.com>wrote:

> Hi George,
>
> thanks for you reply and sorry for the silly mistake.
>
> I change the hbase-site.xml to this:
>
> <configuration>
>     <configuration>
>         <property>
>             <name>hbase.cluster.distributed</name>
>             <value>true</value>
>         </property>
>         <property>
>             <name>hbase.rootdir</name>
>             <value>hdfs://dev32:9000/hbase</value>
>         </property>
>         <property>
>             <name>hbase.zookeeper.quorum</name>
>             <value>dev32</value>
>         </property>
>     </configuration>
> </configuration>
>
> and sync it with slave nodes. but I am still getting the same problem.
>
> I also run the flume instance on the same cluster. Do the flume and hbase
> share the same zookeeper? Is this the reason why I get this problem?
>
>
>
>
> On Mon, Dec 6, 2010 at 7:27 PM, Lars George <lars.george@gmail.com> wrote:
>
>> Hi Exception,
>>
>> For starters the logs say you are trying the wrong ZooKeeper node to
>> get the HBase details (localhost) and you config has:
>>
>>  <property>
>>       <name>hbase.zookeeper.quorum</name>
>>       <value>dev32</value>
>>  </property>
>>  <property>
>>   <name>hbase.zookeeper.quorum</name>
>>   <value>localhost</value>
>>  </property>
>> <property>
>>
>> You are declaring it twice and the last one wins. Remove the second
>> definition.
>>
>> Lars
>>
>> On Mon, Dec 6, 2010 at 8:55 AM, exception qin <exceptionqin2@gmail.com>
>> wrote:
>> > hello,
>> >
>> > I have written a simple java program to load data and scan table in
>> hbase,
>> > but get problems.
>> >
>> > this is my code snap:
>> >
>> > HBaseConfiguration config = new HBaseConfiguration();
>> > HTable table = new HTable(config, "logtable");
>> > ......
>> > //do something with the table
>> >
>> > this program blocked when trying to construct the HTable. here is the
>> log:
>> >
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> > environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14
>> GMT
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:
>> host.name
>> > =dev32
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> > environment:java.version=1.6.0_12
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> > environment:java.vendor=Sun Microsystems Inc.
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> > environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.12/jre
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> >
>> environment:java.class.path=/root/hadoop//hadoop-0.20.2-core.jar:/root/hadoop//lib/log4j-1.2.15.jar:/root/hadoop//lib/commons-logging-1.0.4.jar:/root/hbase/hbase-0.20.6.jar:/root/zookeeper/zookeeper-3.3.1.jar:.
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> >
>> environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> > environment:java.io.tmpdir=/tmp
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> > environment:java.compiler=<NA>
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:os.name
>> =Linux
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> environment:os.arch=amd64
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> > environment:os.version=2.6.26-2-amd64
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:
>> user.name
>> > =root
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> > environment:user.home=/root
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>> > environment:user.dir=/root/eqin/hbaseclient/bin
>> > 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Initiating client
>> connection,
>> > connectString=localhost:2181 sessionTimeout=60000
>> >
>> watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher@47df280b
>> > 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Opening socket connection
>> to
>> > server localhost/127.0.0.1:2181
>> > 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Socket connection
>> established
>> > to localhost/127.0.0.1:2181, initiating session
>> > 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Session establishment
>> complete
>> > on server localhost/127.0.0.1:2181, sessionid = 0x12cba604b020005,
>> > negotiated timeout = 40000
>> >
>> >
>> > I am on hbase 0.20.6 and I have set up a fully distributed cluster with
>> > three nodes.(dev32,dev_26,vmtest).
>> > dev32 is the master node, on which the zookeeper and hmaster located.
>> > there are two region servers running on dev_26 and vmtest. I tested the
>> > hbase shell, it works fine. probably, there is something wrong with my
>> > config. so I print some of my config file.
>> >
>> > zoo.cfg
>> >
>> > # The number of milliseconds of each tick
>> > tickTime=2000
>> > # The number of ticks that the initial
>> > # synchronization phase can take
>> > initLimit=10
>> > # The number of ticks that can pass between
>> > # sending a request and getting an acknowledgement
>> > syncLimit=5
>> > # the directory where the snapshot is stored.
>> > dataDir=/var/zookeeper
>> > # the port at which the clients will connect
>> > clientPort=2181
>> > server.0=dev32:2888:3888
>> >
>> > hbase-site.xnl
>> >
>> > ......
>> >  <property>
>> >        <name>hbase.zookeeper.quorum</name>
>> >        <value>dev32</value>
>> >  </property>
>> >  <property>
>> >    <name>hbase.zookeeper.quorum</name>
>> >    <value>localhost</value>
>> >  </property>
>> > <property>
>> >    <name>hbase.zookeeper.property.clientPort</name>
>> >    <value>2181</value>
>> > </property>
>> > ......
>> >
>> > I have found a lot of sample code. But it doesn't help.  What should I
>> do to
>> > let my program run? Any help on these issues would be greatly
>> appreciated.
>> >
>> >
>> >
>> > Cheers
>> > Exception
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message