hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From exception qin <exceptionq...@gmail.com>
Subject Re: blocked when creating HTable
Date Tue, 07 Dec 2010 10:25:02 GMT
thanks Geogre.

I have already shut down the Flume instance. So there should be no zookeeper
conflict.

I wrote a shell script to run the java program. This is the script:

#!/bin/bash

HADOOPHOME="/root/hadoop/";
HBASEHOME="/root/hbase";
ZOOKEEPERHOME="/root/zookeeper";

RUNLIB="${HADOOPHOME}/hadoop-0.20.2-core.jar:${HADOOPHOME}/lib/log4j-1.2.15.jar:${HADOOPHOME}/lib/commons-logging-1.0.4.jar:$
{HBASEHOME}/hbase-0.20.6.jar:${ZOOKEEPERHOME}/zookeeper-3.3.1.jar:.";

sudo java -classpath ${RUNLIB} org/taomee/HBaseClient;

For the config problem, I set them manually in the code. Now the log looks
like:

10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14 GMT
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client environment:host.name
=dev32
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_12
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.12/jre
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
environment:java.class.path=/root/hadoop//hadoop-0.20.2-core.jar:/root/hadoop//lib/log4j-1.2.15.jar:/root/hadoop//lib/commons-logging-1.0.4.jar:/root/hbase/hbase-0.20.6.jar:/root/zookeeper/zookeeper-3.3.1.jar:.
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/tmp
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
environment:os.version=2.6.26-2-amd64
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client environment:user.name
=root
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
environment:user.home=/root
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
environment:user.dir=/root/eqin/hbaseclient/bin
10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=dev32:2181 sessionTimeout=60000
watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher@bbe0f0a
10/12/07 18:08:09 INFO zookeeper.ClientCnxn: Opening socket connection to
server dev32/10.1.1.32:2181
10/12/07 18:08:09 INFO zookeeper.ClientCnxn: Socket connection established
to dev32/10.1.1.32:2181, initiating session
10/12/07 18:08:09 INFO zookeeper.ClientCnxn: Session establishment complete
on server dev32/10.1.1.32:2181, sessionid = 0x12cc00271d00013, negotiated
timeout = 40000

As you can see, it can connect to zookeeper but still block.

I dig into the code a little bit and find the program blocked at this line:
this.connection.locateRegion(tableName, HConstants.EMPTY_START_ROW);
at  HTable.java

This problem should be related to the configuration. Is there anything else
I need to do?

thanks.

On Tue, Dec 7, 2010 at 4:32 PM, Lars George <lars.george@gmail.com> wrote:

> Hi Exception,
>
> This is up to you to set up properly. If you run them on the same
> cluster/network then you either share the same ZooKeeper or make sure they
> use different ports (as per the zoo.cfg). Also make sure you have the proper
> ZooKeeper quorum set and your client being able to "see" it. If you see
> something like this on you client
>
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Initiating client
> connection,
> >>> connectString=localhost:2181 sessionTimeout=60000
>
> It means that your client does *not* read the proper config but uses the
> defaults. What is the classpath you use on the client?
>
> Lars
>
> On Dec 7, 2010 at 2:34, exception qin <exceptionqin2@gmail.com> wrote:
>
> > Hi George,
> >
> > thanks for you reply and sorry for the silly mistake.
> >
> > I change the hbase-site.xml to this:
> >
> > <configuration>
> >    <configuration>
> >        <property>
> >            <name>hbase.cluster.distributed</name>
> >            <value>true</value>
> >        </property>
> >        <property>
> >            <name>hbase.rootdir</name>
> >            <value>hdfs://dev32:9000/hbase</value>
> >        </property>
> >        <property>
> >            <name>hbase.zookeeper.quorum</name>
> >            <value>dev32</value>
> >        </property>
> >    </configuration>
> > </configuration>
> >
> > and sync it with slave nodes. but I am still getting the same problem.
> >
> > I also run the flume instance on the same cluster. Do the flume and hbase
> > share the same zookeeper? Is this the reason why I get this problem?
> >
> >
> >
> >
> > On Mon, Dec 6, 2010 at 7:27 PM, Lars George <lars.george@gmail.com>
> wrote:
> >
> >> Hi Exception,
> >>
> >> For starters the logs say you are trying the wrong ZooKeeper node to
> >> get the HBase details (localhost) and you config has:
> >>
> >> <property>
> >>      <name>hbase.zookeeper.quorum</name>
> >>      <value>dev32</value>
> >> </property>
> >> <property>
> >>  <name>hbase.zookeeper.quorum</name>
> >>  <value>localhost</value>
> >> </property>
> >> <property>
> >>
> >> You are declaring it twice and the last one wins. Remove the second
> >> definition.
> >>
> >> Lars
> >>
> >> On Mon, Dec 6, 2010 at 8:55 AM, exception qin <exceptionqin2@gmail.com>
> >> wrote:
> >>> hello,
> >>>
> >>> I have written a simple java program to load data and scan table in
> >> hbase,
> >>> but get problems.
> >>>
> >>> this is my code snap:
> >>>
> >>> HBaseConfiguration config = new HBaseConfiguration();
> >>> HTable table = new HTable(config, "logtable");
> >>> ......
> >>> //do something with the table
> >>>
> >>> this program blocked when trying to construct the HTable. here is the
> >> log:
> >>>
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>> environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14
> GMT
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:
> host.name
> >>> =dev32
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.version=1.6.0_12
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.vendor=Sun Microsystems Inc.
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.12/jre
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>
> >>
> environment:java.class.path=/root/hadoop//hadoop-0.20.2-core.jar:/root/hadoop//lib/log4j-1.2.15.jar:/root/hadoop//lib/commons-logging-1.0.4.jar:/root/hbase/hbase-0.20.6.jar:/root/zookeeper/zookeeper-3.3.1.jar:.
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>
> >>
> environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.io.tmpdir=/tmp
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>> environment:java.compiler=<NA>
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:os.name
> >> =Linux
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >> environment:os.arch=amd64
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>> environment:os.version=2.6.26-2-amd64
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:
> user.name
> >>> =root
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>> environment:user.home=/root
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>> environment:user.dir=/root/eqin/hbaseclient/bin
> >>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Initiating client
> connection,
> >>> connectString=localhost:2181 sessionTimeout=60000
> >>>
> >>
> watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher@47df280b
> >>> 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Opening socket connection
> to
> >>> server localhost/127.0.0.1:2181
> >>> 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Socket connection
> >> established
> >>> to localhost/127.0.0.1:2181, initiating session
> >>> 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Session establishment
> >> complete
> >>> on server localhost/127.0.0.1:2181, sessionid = 0x12cba604b020005,
> >>> negotiated timeout = 40000
> >>>
> >>>
> >>> I am on hbase 0.20.6 and I have set up a fully distributed cluster with
> >>> three nodes.(dev32,dev_26,vmtest).
> >>> dev32 is the master node, on which the zookeeper and hmaster located.
> >>> there are two region servers running on dev_26 and vmtest. I tested the
> >>> hbase shell, it works fine. probably, there is something wrong with my
> >>> config. so I print some of my config file.
> >>>
> >>> zoo.cfg
> >>>
> >>> # The number of milliseconds of each tick
> >>> tickTime=2000
> >>> # The number of ticks that the initial
> >>> # synchronization phase can take
> >>> initLimit=10
> >>> # The number of ticks that can pass between
> >>> # sending a request and getting an acknowledgement
> >>> syncLimit=5
> >>> # the directory where the snapshot is stored.
> >>> dataDir=/var/zookeeper
> >>> # the port at which the clients will connect
> >>> clientPort=2181
> >>> server.0=dev32:2888:3888
> >>>
> >>> hbase-site.xnl
> >>>
> >>> ......
> >>> <property>
> >>>       <name>hbase.zookeeper.quorum</name>
> >>>       <value>dev32</value>
> >>> </property>
> >>> <property>
> >>>   <name>hbase.zookeeper.quorum</name>
> >>>   <value>localhost</value>
> >>> </property>
> >>> <property>
> >>>   <name>hbase.zookeeper.property.clientPort</name>
> >>>   <value>2181</value>
> >>> </property>
> >>> ......
> >>>
> >>> I have found a lot of sample code. But it doesn't help.  What should I
> do
> >> to
> >>> let my program run? Any help on these issues would be greatly
> >> appreciated.
> >>>
> >>>
> >>>
> >>> Cheers
> >>> Exception
> >>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message