hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ramkrishna S Vasudevan <ramkrishna.vasude...@huawei.com>
Subject RE: hbase RegionServer dead suddenly
Date Mon, 02 Jan 2012 08:34:02 GMT
If MemStore flush fails RS aborts.  As per the below stack trace I find that
the flushing was not 
Successful due to some failure in HDFS.  Can you check the NN and DN logs at
this time?

Regards
Ram


-----Original Message-----
From: xiaochao [mailto:cschy.2008@163.com] 
Sent: Saturday, December 31, 2011 9:02 AM
To: user@hbase.apache.org
Subject: hbase RegionServer dead suddenly

Hello:

hbase RegionServer is dead with the log below.can someone help me?

2011-12-28 12:12:34,899 FATAL 
org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region 
server serverName=192.168.3.39,50012,1324979072340, load=
(requests=403, regions=1398, usedHeap=3731, maxHeap=9983): Replay of 
HLog required. Forcing server shutdown
org.apache.hadoop.hbase.DroppedSnapshotException: region: 
guard_tb_20111226,Szzzzzzzzzzz,1322710771589.89644afd8e35f06181706e87
4747129e.
        at 
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HReg
ion.java:995)
        at 
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HReg
ion.java:900)
        at 
org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java
:852)
        at 
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(Mem
StoreFlusher.java:392)
        at 
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlob
alPressure(MemStoreFlusher.java:200)
        at 
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlu
sher.java:220)
Caused by: java.io.IOException: Bad connect ack with firstBadLink as 
192.168.3.37:50001
        at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.createBlockOutputSt
ream(DFSClient.java:3139)
        at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStre
am(DFSClient.java:3055)
        at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1900(DFSClie
nt.java:2305)
        at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DF
SClient.java:2500)


Mime
View raw message