hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ChingShen <chingshenc...@gmail.com>
Subject Long GC pause question
Date Thu, 06 Jan 2011 07:44:32 GMT
Hi all,

   I encounter a problem about long gc pause cause the region server's local
zookeeper client cannot send heartbeats, the session times out.
 But I want to know why the HBase master sends a MSG_REGIONSERVER_STOP op to
region sever to stop its services rather than reinitialize a new zookeeper
client or restart region server?

  There are 3 RS/DN/TT and 1 MS/NN/JT in my cluster(Hadoop-0.20.2,  HBase
0.20.6), and set vm.swappiness to zero.

hbase-ites-master-clusterPC1.log
2011-01-06 13:10:57,003 INFO
org.apache.hadoop.hbase.master.ServerManager:
*clusterPC4,60020,1294280765301
znode expired*
2011-01-06 13:10:57,004 DEBUG org.apache.hadoop.hbase.master.HMaster:
Processing todo: ProcessServerShutdown of
ites-clusterPC4,60020,1294280765301
2011-01-06 13:10:57,004 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: process shutdown of
server clusterPC4,60020,1294280765301: logSplit: false, rootRescanned:
false, numberOfMetaRegions: 1, onlineMetaRegions.size(): 1
2011-01-06 13:10:57,007 INFO org.apache.hadoop.hbase.regionserver.HLog:
Splitting 1 hlog(s) in
hdfs://clusterPC1:54001/hbase20_6/.logs/ites-clusterPC4,60020,1294280765301
2011-01-06 13:10:57,007 DEBUG org.apache.hadoop.hbase.regionserver.HLog:
Splitting hlog 1 of 1:
hdfs://clusterPC1:54001/hbase20_6/.logs/ites-clusterPC4,60020,1294280765301/hlog.dat.1294280765667,
length=0
.............

hbase-ites-regionserver-clusterPC4.log:
2011-01-06 12:21:15,773 DEBUG
org.apache.hadoop.hbase.regionserver.LogRoller: Hlog roll period 3600000ms
elapsed
*2011-01-06 13:11:03,849 WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x12d59208f560000 to sun.nio.ch.SelectionKeyImpl@402f0df1
java.io.IOException: TIMED OUT
        at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:906)
2011-01-06 13:11:09,628 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGIONSERVER_STOP*
2011-01-06 13:11:31,491 INFO org.apache.hadoop.ipc.HBaseServer: Stopping
server on 60020
............

Please see the attach files.
Thanks.

Shen

Mime
View raw message