hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tianying Chang <tich...@ebaysf.com>
Subject NullPointerException while loading large amount of new rows into HBase, exception is thrown when trying to obtain lock for RowKey
Date Thu, 16 May 2013 17:12:48 GMT

When our customers(using TSDB) loads large amount of data into HBase, we saw many NullPointerException
in the RS logs as below. I checked the source code, it seems when trying to obtain the lock
for a rowKey, if the entry for that row already exists and the "waitfoBlock" is false (therefore
it won't retry, but just return a NULL value). I can see in doMiniBatchMutation(), the waitForBlock
is set to be false. (most other places "waitForBlock" is always set to true).

This exception is thrown from function lockRow(), which has been deprecated. I am not sure
why it is deprecated, and what is used to replace it. Is this normal? which implies the HBase
should not throw this misleading error message to log.  Or should the client call some other


2013-05-14 12:45:30,911 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Row lock
-3430274391270203797 explicitly acquired by client
2013-05-14 12:45:30,911 WARN org.apache.hadoop.ipc.HBaseServer: (responseTooSlow): {"processingtimems":29783,"call":"lockRow([B@339a6a5c,
[B@5ebcd87b), rpc version=1, client version=29, methodsFingerPrint=0","client":"","starttimems":1368560701128,"queuetimems":847,"class":"HRegionServer","responsesize":0,"method":"lockRow"}
2013-05-14 12:46:00,911 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining
row lock (fsOk: true)
                at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:881)
                at org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:2346)
                at org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:2332)
                at sun.reflect.GeneratedMethodAccessor156.invoke(Unknown Source)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                at java.lang.reflect.Method.invoke(Method.java:597)
                at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:384)
                at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
2013-05-14 12:46:02,514 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server Responder, call
get([B@2166c821, {"timeRange":[0,9223372036854775807],"totalColumns":1,"cacheBlocks":true,"families":{"id":["tagv"]},"maxVersions":1,"row":"slcsn-s00314.slc.ebay.com"}),
rpc version=1, client version=29, methodsFingerPrint=0 from output error
2013-05-14 12:46:02,514 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60020
caught: java.io.IOException: Connection reset by peer
                at sun.nio.ch.FileDispatcher.write0(Native Method)
                at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message