hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10015) Major performance improvement: Avoid synchronization in StoreScanner
Date Mon, 25 Nov 2013 19:30:37 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13831814#comment-13831814
] 

Lars Hofhansl commented on HBASE-10015:
---------------------------------------

So it seems this is universal.
Our performance dude has actually confirmed that this is an expected outcome. I'll gather
some more CPU metrics as I find time today.

[~stack], these locks are taken in StoreScanner, which is "row" based (unlike StoreFileScanner
and MemstoreScanner). So the frequent locks/unlocks there we'd mostly see per row. Wider rows
won't be slower, but the speedup effect would be proportionally less. To be sure I'll validate
with wider tables (maybe 20 CQs). I will also double check that biased locking is in effect.


> Major performance improvement: Avoid synchronization in StoreScanner
> --------------------------------------------------------------------
>
>                 Key: HBASE-10015
>                 URL: https://issues.apache.org/jira/browse/HBASE-10015
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>         Attachments: 10015-0.94-lock.txt, 10015-0.94-new-sample.txt, 10015-0.94-v2.txt,
10015-0.94-v3.txt, 10015-0.94-v4.txt, 10015-0.94-withtest.txt, 10015-0.94.txt, 10015-trunk-v2.txt,
10015-trunk-v3.txt, 10015-trunk-v4.txt, 10015-trunk-v4.txt, 10015-trunk-v4.txt, 10015-trunk.txt,
TestLoad.java
>
>
> Did some more profiling (this time with a sampling profiler) and StoreScanner.peek()
showed up a lot in the samples. At first that was surprising, but peek is synchronized, so
it seems a lot of the sync'ing cost is eaten there.
> It seems the only reason we have to synchronize all these methods is because a concurrent
flush or compaction can change the scanner stack, other than that only a single thread should
access a StoreScanner at any given time.
> So replaced updateReaders() with some code that just indicates to the scanner that the
readers should be updated and then make it the using thread's responsibility to do the work.
> The perf improvement from this is staggering. I am seeing somewhere around 3x scan performance
improvement across all scenarios.
> Now, the hard part is to reason about whether this is 100% correct. I ran TestAtomicOperation
and TestAcidGuarantees a few times in a loop, all still pass.
> Will attach a sample patch.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message