hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yves Langisch <y...@langisch.ch>
Subject Re: NPE while obtaining row lock
Date Tue, 10 Jan 2012 15:51:52 GMT
Still happens with HBase 0.90.5/Hadoop 1.0.0. But I think I have some more insights on this
topic. Following an up to date stack trace:

java.lang.NullPointerException
        at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:986)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:2008)
        at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
Caused by: java.lang.NullPointerException
        at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:881)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:2018)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:2004)
        ... 5 more

After checking the source code I've noticed that the value which is going to be put into the
HashMap can be null in the case where the waitForLock flag is true or the rowLockWaitDuration
is expired (HRegion#internalObtainRowLock, line 2111ff). The latter I think happens in our
case as we have heavy load hitting the server.

IMHO this case should be handled somehow and must not lead to a NPE.

-
Yves

On Dec 30, 2011, at 12:12 PM, Yves Langisch wrote:

> Still happens but before I'm going to add some debugging information I'll try to deploy
the new version 0.90.5.
> 
> -
> Yves
> 
> On Dec 18, 2011, at 12:08 AM, Stack wrote:
> 
>> On Fri, Dec 16, 2011 at 8:20 AM, Yves Langisch <yves@langisch.ch> wrote:
>>> I'm using the async hbase client (1.0) and there is no way to choose a lockId
on my own:
>>> 
>>> <snippet>
>>> return database.client().lockRow(
>>>                   new RowLockRequest(TableManager.ID_TABLE_NAME, MAXID_ROW)).join();
>>> 
>>> </snippet>
>>> 
>>> Any ideas what else could be wrong here?
>>> 
>> 
>> Looking at the code on regionserver side,
>> http://svn.apache.org/viewvc/hbase/tags/0.90.4/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java?view=markup,
>> down around line 1994, its unlikely the region is null since we should
>> throw NotServingRegionException if can't find region (and we check for
>> null region name a few lines up) so maybe its something in the way we
>> do obtainRowLock on line 1995?
>> 
>> Any chance of your instrumenting the regionserver?  Adding a bit of
>> debugging and deploying the debugging regionserver?
>> 
>> My guess is we haven't seen this before because not many use rowlocks
>> (rowlocks if long-lived and lots of contending clients could freeze
>> you out of the server; each client blocked waiting on rowlock to clear
>> occupies a handler of which there are a bounded number).
>> 
>> St.Ack
>> 
> 


Mime
View raw message