hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
Date Tue, 24 Feb 2015 06:39:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14334491#comment-14334491

Lars Hofhansl commented on HBASE-13082:

Wait. A. Minute... The scenario that I describe above already happens with the current code.
StoreScanner.next(List<Cells>) would loop - with the look held - until either we found
a row worth of data or exhausted the entire store. If the region has only one CF that could
mean the entire region is scanned.

So my change would not make it much worse, but cause any other stores of the region not be
able to flush/compact during that time.

(I also see what the issue with the comparison is, if the ts of the cell falls before the
minStamp of the we seek to the next column... We'll do this over and over again. But that's
for a different jira).

So back to this.
* Advantage: Much better scan performance, that can be even measure in 25% higher disk read
rate. (will try with rotating disks tomorrow)
* Disadvantage: a slow scan that does not match any cell in *any* store (CF) can prevent *other*
stores in the region from flushing/compacting until the slow scan finished.

Worth doing?

> Coarsen StoreScanner locks to RegionScanner
> -------------------------------------------
>                 Key: HBASE-13082
>                 URL: https://issues.apache.org/jira/browse/HBASE-13082
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>         Attachments: 13082.txt
> Continuing where HBASE-10015 left of.
> We can avoid locking (and memory fencing) inside StoreScanner by deferring to the lock
already held by the RegionScanner.
> In tests this shows quite a scan improvement and reduced CPU (the fences make the cores
wait for memory fetches).
> There are some drawbacks too:
> * All calls to RegionScanner need to be remain synchronized
> * Implementors of coprocessors need to be diligent in following the locking contract.
For example Phoenix does not lock RegionScanner.nextRaw() and required in the documentation
(not picking on Phoenix, this one is my fault as I told them it's OK)
> * possible starving of flushes and compaction with heavy read load. RegionScanner operations
would keep getting the locks and the flushes/compactions would not be able finalize the set
of files.
> I'll have a patch soon.

This message was sent by Atlassian JIRA

View raw message