hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1311) ZooKeeperWrapper: Failed to set watcher on ZNode /hbase/master
Date Thu, 07 May 2009 17:25:31 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707012#action_12707012
] 

stack commented on HBASE-1311:
------------------------------

Here is what I saw on master side:

{code}
2009-05-07 07:40:05,597 [RegionManager.rootScanner-EventThread] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Got ZooKeeper event, state: SyncConnected, type: None, path: null
2009-05-07 07:40:07,555 [RegionManager.rootScanner] INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.rootScanner scan of 1 row(s) of meta region {regionname: -ROOT-,,0, startKey:
<>, server: 208.76.44.142:60021} complete
2009-05-07 07:40:19,983 [IPC Server handler 6 on 60000] DEBUG org.apache.hadoop.hbase.master.ServerManager:
Total Load: 1399, Num Servers: 3, Avg Load: 467.0
2009-05-07 07:40:27,089 [main-EventThread] INFO org.apache.hadoop.hbase.master.ServerManager:
aa0-000-15.u.powerset.com_1241637114572_60021 znode expired
2009-05-07 07:40:27,120 [HMaster] DEBUG org.apache.hadoop.hbase.master.HMaster: Processing
todo: ProcessServerShutdown of aa0-000-15.u.powerset.com_1241637114572_60021
2009-05-07 07:40:27,120 [HMaster] INFO org.apache.hadoop.hbase.master.RegionServerOperation:
process shutdown of server aa0-000-15.u.powerset.com_1241637114572_60021: logSplit: false,
rootRescanned: false, numberOfMetaRegions: 1, onlineMetaRegions.size(): 1
2009-05-07 07:40:27,125 [HMaster] INFO org.apache.hadoop.hbase.regionserver.HLog: Splitting
3 log(s) in hdfs://aa0-000-12.u.powerset.com:9000/hbasetrunk2/@LOGS@/aa0-000-15.u.powerset.com_1241637114572_60021
2009-05-07 07:40:27,126 [HMaster] DEBUG org.apache.hadoop.hbase.regionserver.HLog: Splitting
1 of 3: hdfs://aa0-000-12.u.powerset.com:9000/hbasetrunk2/@LOGS@/aa0-000-15.u.powerset.com_1241637114572_60021/hlog.dat.1241676102482
2009-05-07 07:40:30,022 [main-EventThread] INFO org.apache.hadoop.hbase.master.ServerManager:
aa0-000-14.u.powerset.com_1241637114470_60021 znode expired
2009-05-07 07:40:34,779 [IPC Server handler 7 on 60000] INFO org.apache.hadoop.ipc.HBaseServer:
IPC Server handler 7 on 60000, call regionServerReport(address: 208.76.44.141:60021, startcode:
1241637114470, load: (requests=0, regions=466, usedHeap=861, maxHeap=1255), [Lorg.apache.hadoop.hbase.HMsg;@191ea7e5,
[Lorg.apache.hadoop.hbase.HRegionInfo;@2ba0b845) from 208.76.44.141:5009
4: error: org.apache.hadoop.hbase.Leases$LeaseStillHeldException
org.apache.hadoop.hbase.Leases$LeaseStillHeldException
        at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:201)
        at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:604)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:642)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:911)
2009-05-07 07:40:34,792 [IPC Server handler 4 on 60000] INFO org.apache.hadoop.ipc.HBaseServer:
IPC Server handler 4 on 60000, call regionServerReport(address: 208.76.44.141:60021, startcode:
1241637114470, load: (requests=0, regions=466, usedHeap=861, maxHeap=1255), [Lorg.apache.hadoop.hbase.HMsg;@5b7836c8,
[Lorg.apache.hadoop.hbase.HRegionInfo;@3154b362) from 208.76.44.141:5019
7: error: org.apache.hadoop.hbase.Leases$LeaseStillHeldException
org.apache.hadoop.hbase.Leases$LeaseStillHeldException
        at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:201)
        at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:604)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:642)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:911)
2009-05-07 07:40:34,801 [IPC Server handler 9 on 60000] INFO org.apache.hadoop.ipc.HBaseServer:
IPC Server handler 9 on 60000, call regionServerReport(address: 208.76.44.141:60021, startcode:
1241637114470, load: (requests=0, regions=466, usedHeap=862, maxHeap=1255), [Lorg.apache.hadoop.hbase.HMsg;@2fcd003b,
[Lorg.apache.hadoop.hbase.HRegionInfo;@3bca3a01) from 208.76.44.141:5019
7: error: org.apache.hadoop.hbase.Leases$LeaseStillHeldException
org.apache.hadoop.hbase.Leases$LeaseStillHeldException
        at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:201)
        at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:604)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:642)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:911)
{code}

Above repeats over and over.

> ZooKeeperWrapper: Failed to set watcher on ZNode /hbase/master
> --------------------------------------------------------------
>
>                 Key: HBASE-1311
>                 URL: https://issues.apache.org/jira/browse/HBASE-1311
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: Andrew Purtell
>            Assignee: Nitay Joffe
>             Fix For: 0.20.0
>
>         Attachments: dumb-wrapper-hack.patch
>
>
> After about 12 hours of operation, this repeats over and over in the regionserver log:
> 2009-04-05 19:44:38,445 WARN org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed
to set watcher on ZNode /hbase/master
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session
expired
> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:118)
> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:709)
> 	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.watchMasterAddress(ZooKeeperWrapper.java:235)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.watchMasterAddress(HRegionServer.java:343)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.process(HRegionServer.java:339)
> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:366)
> 2009-04-05 19:44:38,445 WARN org.apache.hadoop.hbase.regionserver.HRegionServer: Unable
to set watcher on ZooKeeper master address. Retrying.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message