hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Coprocessor tax?
Date Tue, 01 Mar 2011 20:25:38 GMT
Looking at the source of
ReentrantReadWriteLock$Sync#tryAcquireShared(), contention on shared
lock acquisition is a little expensive. It's basically equally as
expensive as doing an atomic-increment on a contended counter.

Basically the lock is a bit-field with the top 16 bits being a "shared
count" and the bottom 16 being "exclusive count". Trying to acquire
the lock in shared-mode is a CAS trying to increment shared count
while exclusive-count is 0. So when under contention, some lockers
will race to change the shared-count and have to spin.

-Todd

On Tue, Mar 1, 2011 at 12:08 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
> My own profiling shows that a read write lock can be up to 3-6% of the
> CPU budget in our put/get query path.  Adding another one if not
> necessary would probably not be good.
>
> In fact in the region coprocessor the only thing the write lock is
> used for is the preClose and postClose, but looking in the
> implementation of those methods I don't really get why this is
> necessary.  The write lock ensures single thread access, but there is
> nothing that prevents other threads from calling other methods AFTER
> the postClose?
>
> -ryan
>
> On Tue, Mar 1, 2011 at 12:02 PM, Gary Helmling <ghelmling@gmail.com> wrote:
>> All the CoprocessorHost invocations should be wrapped in "if (cpHost !=
>> null)".  We could just added an extra check for whether any coprocessors are
>> loaded -- "if (cpHost != null && cpHost.isActive())", something like that?
>>  Or the CoprocessorHost methods could do this checking internally.
>>
>> Either way should be relatively easy to bypass the lock acquisition.  Is
>> there much overhead to acquiring a read lock if the write lock is never
>> taken though? (just wondering)
>>
>>
>>
>> On Tue, Mar 1, 2011 at 11:51 AM, Stack <stack@duboce.net> wrote:
>>
>>> So, I'm debugging something else but thread dumping I see a bunch of this:
>>>
>>>
>>> "IPC Server handler 6 on 61020" daemon prio=10 tid=0x00000000422d2800
>>> nid=0x7714 runnable [0x00007f1c5acea000]
>>>   java.lang.Thread.State: RUNNABLE
>>>        at
>>> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:434)
>>>        at
>>> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:404)
>>>        at
>>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1260)
>>>        at
>>> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:594)
>>>        at
>>> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:532)
>>>        at
>>> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchPut(HRegion.java:1476)
>>>        at
>>> org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1454)
>>>        at
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:2652)
>>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>        at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>        at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>>        at
>>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:309)
>>>        at
>>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1060)
>>>
>>>
>>> Do others?  I don't have any CPs loaded.  I'm wondering if we can do
>>> more to just avoid the CP codepath if no CPs loaded.
>>>
>>> St.Ack
>>>
>>
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message