hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tianying Chang <tich...@ebaysf.com>
Subject openTSDB lose large amount of data when the client are writing
Date Thu, 19 Sep 2013 17:09:57 GMT
Hi, 

I have a customer who use openTSDB. Recently we found that only less than 10% data are written,
rest are are lost. By checking the RS log, there are many row lock related issues, like below.
It seems large amount of write to tsdb that need row lock caused the problem. Anyone else
see similar problem?  Is it a bug of openTSDB? Or it is due to HBase exposed a vulnerable
API? 

org.apache.hadoop.hbase.UnknownRowLockException: Invalid row lock
	at org.apache.hadoop.hbase.regionserver.HRegionServer.getLockFromId(HRegionServer.java:2732)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2071)
	at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
13/09/18 12:08:30 ERROR regionserver.HRegionServer: 
org.apache.hadoop.hbase.UnknownRowLockException: -6180307918863136448
	at org.apache.hadoop.hbase.regionserver.HRegionServer.unlockRow(HRegionServer.java:2765)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
	

Thanks
Tian-Ying 

Mime
View raw message