hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vimal Jain <vkj...@gmail.com>
Subject High Full GC count for Region server
Date Tue, 22 Oct 2013 07:02:16 GMT
I am running in Hbase in pseudo distributed mode. ( Hadoop version - 1.1.2
, Hbase version - 0.94.7 )
I am getting few exceptions in both hadoop ( namenode , datanode) logs and
hbase(region server).
When i search for these exceptions on google , i concluded  that problem is
mainly due to large number of full GC in region server process.

I used jstat and found that there are total of 950 full GCs in span of 4
days for region server process.Is this ok?

I am totally confused by number of exceptions i am getting.
Also i get below exceptions intermittently.

Region server:-

2013-10-22 12:00:26,627 WARN org.apache.hadoop.ipc.HBaseServer:
{"processingtimems":15312,"call":"next(-6681408251916104762, 1000), rpc
version=1, client version=29, methodsFingerPrint=-1368823753","client":"
2013-10-22 12:06:17,606 WARN org.apache.hadoop.ipc.HBaseServer:
(operationTooSlow): {"processingtimems":14759,"client":"

2013-10-18 10:37:45,008 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer
Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
could only be replicated to 0 nodes, instead of 1

Name node :-
java.io.IOException: File
could only be replicated to 0 nodes, instead of 1

java.io.IOException: Got blockReceived message from unregistered or dead
node blk_-2949905629769882833_52274

Data node :-
480000 millis timeout while waiting for channel to be ready for write. ch :
java.nio.channels.SocketChannel[connected local=/]

ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(,
storageID=DS-1816106352-, infoPort=50075,
java.io.EOFException: while trying to read 39309 bytes

Thanks and Regards,
Vimal Jain

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message