Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of exceptionqin2@gmail.com
 designates 209.85.212.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=eH/dVbKnEJ3MD4Wi6gzPWkNaIRoRNKjP4LxDR55gyMadcgRprNKo8+caPPO27cCfq/
         OkH/8rG92bCcfEOge/DrbMUjvZOYfyVQvHm4RsTWwsyeqNVJWSmAi73NGRQxdBtyaH0r
         EJQfqT2J11eSHsmskl391A3h+fY3CqZwIHf60=
MIME-Version: 1.0
In-Reply-To: <31807969-7A4D-4706-B739-272432753E2E@gmail.com>
References: <AANLkTi=vcFX1St0-JHzL+nrB2OkRH7VHOnRcWwKYrd3t@mail.gmail.com>
	<AANLkTi=U6iOqUJLhdnzcb61F5Tzka65k25j7eyGSU8=V@mail.gmail.com>
	<AANLkTikYzmzWSQrtb7nLSyoV2FA8qQAD8fDzhPtyYDPG@mail.gmail.com>
	<0EE3165D-B373-45C6-9B79-D4A62C63DE4A@gmail.com>
	<AANLkTi=ek3Uzr_61WdXJzfgJQm+rHB52Nc2fK146YArv@mail.gmail.com>
	<31807969-7A4D-4706-B739-272432753E2E@gmail.com>
Date: Tue, 7 Dec 2010 19:51:07 +0800
Message-ID: <AANLkTim_AA3Zr7xha71a4F25vESH+nt6_R0TfCOBEAnq@mail.gmail.com>
Subject: Re: blocked when creating HTable
From: exception qin <exceptionqin2@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=000e0cd3285cb834820496d09e62

--000e0cd3285cb834820496d09e62
Content-Type: text/plain; charset=ISO-8859-1

yes, the hbase shell works fine.I can scan both '-ROOT-' and '.META.'. Also,
I created a table and put some data into it.

The port 60020 is listened by region server:
tcp6       0      0 dev_26:60020    [::]:*   LISTEN      hbase      1563816
    5610/java

I wrote a simple program to try to locate region for root table, but got the
following exception:

Exception in thread "main" java.io.IOException: Call to dev_26/
10.1.1.26:60020 failed on local exception: java.io.EOFException
        at
org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:770)
        at
org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:740)
        at
org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:333)
        at $Proxy0.getProtocolVersion(Unknown Source)
        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:489)
        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:465)
        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:512)
        at
org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:423)
        at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:918)
        at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:934)
        at org.taomee.HBaseClient.main(Unknown Source)
Caused by: java.io.EOFException
        at java.io.DataInputStream.readInt(DataInputStream.java:375)
        at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:506)
        at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:450)

Does this mean port 60020 unaccessable?


here is the code:

        HBaseConfiguration config = new HBaseConfiguration();
        config.set("hbase.zookeeper.quorum","dev32");
        config.set("hbase.cluster.distributed","true");
        config.set("hbase.rootdir","hdfs://dev32:9000/hbase");

        HConnection connection = HConnectionManager.getConnection(config);
        byte [] metaKey =
HRegionInfo.createRegionName(Bytes.toBytes(tablename),
HConstants.EMPTY_START_ROW,HConstants.NINES);

        ZooKeeperWrapper zk =
HConnectionManager.getClientZooKeeperWatcher(config).getZooKeeperWrapper();
        HServerAddress rootRegionAddress = zk.readRootRegionLocation();
        if (rootRegionAddress==null)
        {
            System.out.println("cannot find root region location");
        }

        HRegionInterface server =
connection.getHRegionConnection(rootRegionAddress);
        server.getRegionInfo(HRegionInfo.ROOT_REGIONINFO.getRegionName());

        connection.locateRegion(HConstants.ROOT_TABLE_NAME, metaKey);


On Tue, Dec 7, 2010 at 6:48 PM, Lars George <lars.george@gmail.com> wrote:

> OK, so ZooKeeper works now and the client can obviously connect. After the
> cluster is running can you start the HBase shell and see if you can scan
> meta or root? Simply try a "> scan '-ROOT-'" in the shell.
>
> Do you have anything blocking access to the server hosting those regions?
> Is port 60020 accessible by the client?
>
> Lars
>
> On Dec 7, 2010, at 11:25, exception qin <exceptionqin2@gmail.com> wrote:
>
> > thanks Geogre.
> >
> > I have already shut down the Flume instance. So there should be no
> zookeeper
> > conflict.
> >
> > I wrote a shell script to run the java program. This is the script:
> >
> > #!/bin/bash
> >
> > HADOOPHOME="/root/hadoop/";
> > HBASEHOME="/root/hbase";
> > ZOOKEEPERHOME="/root/zookeeper";
> >
> >
> RUNLIB="${HADOOPHOME}/hadoop-0.20.2-core.jar:${HADOOPHOME}/lib/log4j-1.2.15.jar:${HADOOPHOME}/lib/commons-logging-1.0.4.jar:$
> > {HBASEHOME}/hbase-0.20.6.jar:${ZOOKEEPERHOME}/zookeeper-3.3.1.jar:.";
> >
> > sudo java -classpath ${RUNLIB} org/taomee/HBaseClient;
> >
> > For the config problem, I set them manually in the code. Now the log
> looks
> > like:
> >
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> > environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14 GMT
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client environment:host.name
> > =dev32
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> > environment:java.version=1.6.0_12
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> > environment:java.vendor=Sun Microsystems Inc.
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> > environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.12/jre
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> >
> environment:java.class.path=/root/hadoop//hadoop-0.20.2-core.jar:/root/hadoop//lib/log4j-1.2.15.jar:/root/hadoop//lib/commons-logging-1.0.4.jar:/root/hbase/hbase-0.20.6.jar:/root/zookeeper/zookeeper-3.3.1.jar:.
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> >
> environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> > environment:java.io.tmpdir=/tmp
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> > environment:java.compiler=<NA>
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client environment:os.name
> =Linux
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:os.arch=amd64
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> > environment:os.version=2.6.26-2-amd64
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client environment:user.name
> > =root
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> > environment:user.home=/root
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> > environment:user.dir=/root/eqin/hbaseclient/bin
> > 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Initiating client connection,
> > connectString=dev32:2181 sessionTimeout=60000
> >
> watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher@bbe0f0a
> > 10/12/07 18:08:09 INFO zookeeper.ClientCnxn: Opening socket connection to
> > server dev32/10.1.1.32:2181
> > 10/12/07 18:08:09 INFO zookeeper.ClientCnxn: Socket connection
> established
> > to dev32/10.1.1.32:2181, initiating session
> > 10/12/07 18:08:09 INFO zookeeper.ClientCnxn: Session establishment
> complete
> > on server dev32/10.1.1.32:2181, sessionid = 0x12cc00271d00013,
> negotiated
> > timeout = 40000
> >
> > As you can see, it can connect to zookeeper but still block.
> >
> > I dig into the code a little bit and find the program blocked at this
> line:
> > this.connection.locateRegion(tableName, HConstants.EMPTY_START_ROW);
> > at  HTable.java
> >
> > This problem should be related to the configuration. Is there anything
> else
> > I need to do?
> >
> > thanks.
> >
> > On Tue, Dec 7, 2010 at 4:32 PM, Lars George <lars.george@gmail.com>
> wrote:
> >
> >> Hi Exception,
> >>
> >> This is up to you to set up properly. If you run them on the same
> >> cluster/network then you either share the same ZooKeeper or make sure
> they
> >> use different ports (as per the zoo.cfg). Also make sure you have the
> proper
> >> ZooKeeper quorum set and your client being able to "see" it. If you see
> >> something like this on you client
> >>
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Initiating client
> >> connection,
> >>>>> connectString=localhost:2181 sessionTimeout=60000
> >>
> >> It means that your client does *not* read the proper config but uses the
> >> defaults. What is the classpath you use on the client?
> >>
> >> Lars
> >>
> >> On Dec 7, 2010 at 2:34, exception qin <exceptionqin2@gmail.com> wrote:
> >>
> >>> Hi George,
> >>>
> >>> thanks for you reply and sorry for the silly mistake.
> >>>
> >>> I change the hbase-site.xml to this:
> >>>
> >>> <configuration>
> >>>   <configuration>
> >>>       <property>
> >>>           <name>hbase.cluster.distributed</name>
> >>>           <value>true</value>
> >>>       </property>
> >>>       <property>
> >>>           <name>hbase.rootdir</name>
> >>>           <value>hdfs://dev32:9000/hbase</value>
> >>>       </property>
> >>>       <property>
> >>>           <name>hbase.zookeeper.quorum</name>
> >>>           <value>dev32</value>
> >>>       </property>
> >>>   </configuration>
> >>> </configuration>
> >>>
> >>> and sync it with slave nodes. but I am still getting the same problem.
> >>>
> >>> I also run the flume instance on the same cluster. Do the flume and
> hbase
> >>> share the same zookeeper? Is this the reason why I get this problem?
> >>>
> >>>
> >>>
> >>>
> >>> On Mon, Dec 6, 2010 at 7:27 PM, Lars George <lars.george@gmail.com>
> >> wrote:
> >>>
> >>>> Hi Exception,
> >>>>
> >>>> For starters the logs say you are trying the wrong ZooKeeper node to
> >>>> get the HBase details (localhost) and you config has:
> >>>>
> >>>> <property>
> >>>>     <name>hbase.zookeeper.quorum</name>
> >>>>     <value>dev32</value>
> >>>> </property>
> >>>> <property>
> >>>> <name>hbase.zookeeper.quorum</name>
> >>>> <value>localhost</value>
> >>>> </property>
> >>>> <property>
> >>>>
> >>>> You are declaring it twice and the last one wins. Remove the second
> >>>> definition.
> >>>>
> >>>> Lars
> >>>>
> >>>> On Mon, Dec 6, 2010 at 8:55 AM, exception qin <
> exceptionqin2@gmail.com>
> >>>> wrote:
> >>>>> hello,
> >>>>>
> >>>>> I have written a simple java program to load data and scan table in
> >>>> hbase,
> >>>>> but get problems.
> >>>>>
> >>>>> this is my code snap:
> >>>>>
> >>>>> HBaseConfiguration config = new HBaseConfiguration();
> >>>>> HTable table = new HTable(config, "logtable");
> >>>>> ......
> >>>>> //do something with the table
> >>>>>
> >>>>> this program blocked when trying to construct the HTable. here is the
> >>>> log:
> >>>>>
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>>> environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14
> >> GMT
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:
> >> host.name
> >>>>> =dev32
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>>> environment:java.version=1.6.0_12
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>>> environment:java.vendor=Sun Microsystems Inc.
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>>> environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.12/jre
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>>>
> >>>>
> >>
> environment:java.class.path=/root/hadoop//hadoop-0.20.2-core.jar:/root/hadoop//lib/log4j-1.2.15.jar:/root/hadoop//lib/commons-logging-1.0.4.jar:/root/hbase/hbase-0.20.6.jar:/root/zookeeper/zookeeper-3.3.1.jar:.
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>>>
> >>>>
> >>
> environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>>> environment:java.io.tmpdir=/tmp
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>>> environment:java.compiler=<NA>
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:
> os.name
> >>>> =Linux
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>> environment:os.arch=amd64
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>>> environment:os.version=2.6.26-2-amd64
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:
> >> user.name
> >>>>> =root
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>>> environment:user.home=/root
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
> >>>>> environment:user.dir=/root/eqin/hbaseclient/bin
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Initiating client
> >> connection,
> >>>>> connectString=localhost:2181 sessionTimeout=60000
> >>>>>
> >>>>
> >>
> watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher@47df280b
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Opening socket
> connection
> >> to
> >>>>> server localhost/127.0.0.1:2181
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Socket connection
> >>>> established
> >>>>> to localhost/127.0.0.1:2181, initiating session
> >>>>> 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Session establishment
> >>>> complete
> >>>>> on server localhost/127.0.0.1:2181, sessionid = 0x12cba604b020005,
> >>>>> negotiated timeout = 40000
> >>>>>
> >>>>>
> >>>>> I am on hbase 0.20.6 and I have set up a fully distributed cluster
> with
> >>>>> three nodes.(dev32,dev_26,vmtest).
> >>>>> dev32 is the master node, on which the zookeeper and hmaster located.
> >>>>> there are two region servers running on dev_26 and vmtest. I tested
> the
> >>>>> hbase shell, it works fine. probably, there is something wrong with
> my
> >>>>> config. so I print some of my config file.
> >>>>>
> >>>>> zoo.cfg
> >>>>>
> >>>>> # The number of milliseconds of each tick
> >>>>> tickTime=2000
> >>>>> # The number of ticks that the initial
> >>>>> # synchronization phase can take
> >>>>> initLimit=10
> >>>>> # The number of ticks that can pass between
> >>>>> # sending a request and getting an acknowledgement
> >>>>> syncLimit=5
> >>>>> # the directory where the snapshot is stored.
> >>>>> dataDir=/var/zookeeper
> >>>>> # the port at which the clients will connect
> >>>>> clientPort=2181
> >>>>> server.0=dev32:2888:3888
> >>>>>
> >>>>> hbase-site.xnl
> >>>>>
> >>>>> ......
> >>>>> <property>
> >>>>>      <name>hbase.zookeeper.quorum</name>
> >>>>>      <value>dev32</value>
> >>>>> </property>
> >>>>> <property>
> >>>>>  <name>hbase.zookeeper.quorum</name>
> >>>>>  <value>localhost</value>
> >>>>> </property>
> >>>>> <property>
> >>>>>  <name>hbase.zookeeper.property.clientPort</name>
> >>>>>  <value>2181</value>
> >>>>> </property>
> >>>>> ......
> >>>>>
> >>>>> I have found a lot of sample code. But it doesn't help.  What should
> I
> >> do
> >>>> to
> >>>>> let my program run? Any help on these issues would be greatly
> >>>> appreciated.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Cheers
> >>>>> Exception
> >>>>>
> >>>>
> >>
>

--000e0cd3285cb834820496d09e62--