hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Silberstein <silbe...@yahoo-inc.com>
Subject Re: Java client problem w/ zookeeper
Date Wed, 13 Jan 2010 21:04:18 GMT
No, nothing at the master, other than a bunch of NodeExists exceptions in
the zookeeper log.

Something else I've found is that I can add records to the table through the
shell, without problems.  It's only from the java client, on a separate box,
that I'm having problems.

Thanks,
Adam


On 1/13/10 12:08 PM, "Jean-Daniel Cryans" <jdcryans@apache.org> wrote:

> It means the root region likely wasn't assigned, anything in the
> master about it?
> 
> J-D
> 
> On Wed, Jan 13, 2010 at 11:43 AM, Adam Silberstein
> <silberst@yahoo-inc.com> wrote:
>> I have version 0.20.2.  The client requests hang.
>> And, I looked in the region server logs...here is the likely culprit:
>> 
>> java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
>> KeeperErrorCode = NoNode for /hbase/root-region-server
>>        at
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKee
>> perWrapper.java:332)
>>        at
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddress(ZooKeeperWrap
>> per.java:318)
>>        at
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readRootRegionLocation(Zo
>> oKeeperWrapper.java:231)
>>        at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:44
>> 2)
>>        at java.lang.Thread.run(Thread.java:619)
>> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
>> KeeperErrorCode = NoNode for /hbase/root-region-server
>>        at
>> org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
>>        at
>> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>>        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:892)
>>        at
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKee
>> perWrapper.java:328)
>>        ... 4 more
>> java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
>> KeeperErrorCode = NoNode for /hbase/root-region-server
>>        at
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKee
>> perWrapper.java:332)
>>        at
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddress(ZooKeeperWrap
>> per.java:318)
>>        at
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readRootRegionLocation(Zo
>> oKeeperWrapper.java:231)
>>        at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:44
>> 2)
>>        at java.lang.Thread.run(Thread.java:619)
>> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
>> KeeperErrorCode = NoNode for /hbase/root-region-server
>>        at
>> org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
>>        at
>> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>>        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:892)
>>        at
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKee
>> perWrapper.java:328)
>>        ... 4 more
>> 
>> Is there zk state on the region servers?  Maybe when wiping the system, I
>> should have wiped something on them.
>> 
>> Thanks,
>> Adam
>> 
>> On 1/13/10 11:33 AM, "Jean-Daniel Cryans" <jdcryans@apache.org> wrote:
>> 
>>> Those node exists are usually harmless, I would like the see what's on
>>> the HBase region server and master side.
>>> 
>>> You say it cannot do any operation but you don't see exceptions
>>> client-side? It just hangs? Which version is it?
>>> 
>>> Thx,
>>> 
>>> J-D
>>> 
>>> On Wed, Jan 13, 2010 at 11:25 AM, Adam Silberstein
>>> <silberst@yahoo-inc.com> wrote:
>>>> Hi,
>>>> I set up a new Hbase cluster and am seeing an error I haven't seen before.
>>>> When I start up Hbase I have this NodeExists error in the zookeeper log:
>>>> 
>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode
=
>>>> NodeExists
>>>>        at
>>>> 
org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>>>>
s
>>>> or.java:245)
>>>>        at
>>>> 
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>>>>
a
>>>> va:114)
>>>> 2010-01-13 11:10:47,503 WARN
>>>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
>>>> processing sessionid:0x1262918396b0002 type:create cxid:0xc
>>>> zxid:0xfffffffffffffffe txntype:unknown n/a
>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode
=
>>>> NodeExists
>>>>        at
>>>> 
org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>>>>
s
>>>> or.java:245)
>>>>        at
>>>> 
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>>>>
a
>>>> va:114)
>>>> 
>>>> I've read that may be harmless.  But then when I run the java client, it
>>>> cannot do any operations.  And I see a similar NodeExists error in the
>>>> zookeeper log.
>>>> 
>>>> 
>>>> 2010-01-13 11:11:24,661 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>> Connected to /203.104.21.210:36928 lastZxid 0
>>>> 2010-01-13 11:11:24,661 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>> Creating new session 0x1262918396b0004
>>>> 2010-01-13 11:11:24,664 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>> Finished init of 0x1262918396b0004 valid:true
>>>> 2010-01-13 11:11:24,973 WARN
>>>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
>>>> processing sessionid:0x1262918396b0004 type:create cxid:0x2
>>>> zxid:0xfffffffffffffffe txntype:unknown n/a
>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode
=
>>>> NodeExists
>>>>        at
>>>> 
org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>>>>
s
>>>> or.java:245)
>>>>        at
>>>> 
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>>>>
a
>>>> va:114)
>>>> 2010-01-13 11:11:33,088 WARN org.apache.zookeeper.server.NIOServerCnxn:
>>>> Exception causing close of session 0x1262918396b0004 due to
>>>> java.io.IOException: Read error
>>>> 2010-01-13 11:11:33,089 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>> closing session:0x1262918396b0004 NIOServerCnxn:
>>>> java.nio.channels.SocketChannel[connected local=/203.104.21.210:2181
>>>> remote=/203.104.21.210:36928]
>>>> 2010-01-13 11:11:42,709 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>> Connected to /203.104.21.212:54837 lastZxid 0
>>>> 2010-01-13 11:11:42,709 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>> Creating new session 0x1262918396b0005
>>>> 2010-01-13 11:11:42,711 INFO org.apache.zookeeper.server.NIOServerCnxn:
>>>> Finished init of 0x1262918396b0005 valid:true
>>>> 2010-01-13 11:11:42,711 WARN
>>>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when
>>>> processing sessionid:0x1262918396b0005 type:create cxid:0x1
>>>> zxid:0xfffffffffffffffe txntype:unknown n/a
>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode
=
>>>> NodeExists
>>>>        at
>>>> 
org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>>>>
s
>>>> or.java:245)
>>>>        at
>>>> 
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>>>>
a
>>>> va:114)
>>>> 2010-01-13 11:12:06,002 INFO
>>>> org.apache.zookeeper.server.SessionTrackerImpl:
>>>> Expiring session 0x1262918396b0004
>>>> 2010-01-13 11:12:06,002 INFO org.apache.zookeeper.server.ZooKeeperServer:
>>>> Expiring session 0x1262918396b0004
>>>> 2010-01-13 11:12:06,002 INFO
>>>> org.apache.zookeeper.server.PrepRequestProcessor: Processed session
>>>> termination request for id: 0x1262918396b0004
>>>> 
>>>> I've tried wiping the whole setup, including the zookeeper files that Hbase
>>>> puts in /tmp and formatting HDFS.  But I get the same problem again.  I'm
>>>> not seeing any exceptions on the client-side.
>>>> 
>>>> Thanks,
>>>> Adam
>>>> 
>>>> 
>> 
>> 


Mime
View raw message