Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Sat, 23 Nov 2013 07:38:35 +0000 (UTC)
From: "stack (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12680362.1384978258355.18669.1385192315709@arcas>
In-Reply-To: <JIRA.12680362.1384978258355@arcas>
References: <JIRA.12680362.1384978258355@arcas>
Subject: [jira] [Commented] (HBASE-10015) Major performance improvement:
 Avoid synchronization in StoreScanner
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-10015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13830612#comment-13830612 ] 

stack commented on HBASE-10015:
-------------------------------

+ Scans run free for N seconds or nanoseconds since checking this should be cheap(?) and then they go to a checkpoint where they look to see if they should reset.  ChangedReaders blocks until checkpoint has been cleared.
+ Scans check for closing being set on each op (I suppose this check of a volatile would be just as bad as a synchronization).
+ We refcount outstanding scanners and only delete compacted files when refcount goes to zero.  Not sure how we'd switch in flushes unless we prevent the flush happening while ongoing scan (eek).


> Major performance improvement: Avoid synchronization in StoreScanner
> --------------------------------------------------------------------
>
>                 Key: HBASE-10015
>                 URL: https://issues.apache.org/jira/browse/HBASE-10015
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>             Fix For: 0.98.0, 0.96.1, 0.94.15
>
>         Attachments: 10015-0.94-v2.txt, 10015-0.94-v3.txt, 10015-0.94-v4.txt, 10015-0.94-withtest.txt, 10015-0.94.txt, 10015-trunk-v2.txt, 10015-trunk-v3.txt, 10015-trunk-v4.txt, 10015-trunk-v4.txt, 10015-trunk-v4.txt, 10015-trunk.txt, TestLoad.java
>
>
> Did some more profiling (this time with a sampling profiler) and StoreScanner.peek() showed up a lot in the samples. At first that was surprising, but peek is synchronized, so it seems a lot of the sync'ing cost is eaten there.
> It seems the only reason we have to synchronize all these methods is because a concurrent flush or compaction can change the scanner stack, other than that only a single thread should access a StoreScanner at any given time.
> So replaced updateReaders() with some code that just indicates to the scanner that the readers should be updated and then make it the using thread's responsibility to do the work.
> The perf improvement from this is staggering. I am seeing somewhere around 3x scan performance improvement across all scenarios.
> Now, the hard part is to reason about whether this is 100% correct. I ran TestAtomicOperation and TestAcidGuarantees a few times in a loop, all still pass.
> Will attach a sample patch.


--
This message was sent by Atlassian JIRA
(v6.1#6144)