hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
Date Sun, 22 Feb 2015 02:13:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14331989#comment-14331989
] 

stack commented on HBASE-13082:
-------------------------------

How does it work?

We create a RegionScanner per Scan. If we want to change the files that make up a scan, we
need to synchronize on 'this'. Calls to next, close, etc., have synchronized on 'this' so
the change of files can only be done when not nexting or closing?

There may be many outstanding region scans going on.  For the completion of file updates to
complete, all outstanding scanners will need to reach a 'safe point', i.e. post a next or
close call (I suppose close don't count ... so just next).... A big row could take a while
to return...

Sounds good to me.

Its kinda dirty passing lock from RegionScanner down from high level for use at StoreScanner
scope but hey, whatever works. Why were we not able to do this at the StoreScanner scope again?

On compaction and flush being delayed, for flush, we will be in the commit phase when we are
trying to swap in the new file -- post flush from memory but the snapshot of the memstore
is still around and being read from -- so we could hold on to memory pressure a little longer.
 For compaction, yeah, could be reading from many files for a while longer though the compaction
finished.

You done any testing. I like the bit where we remove the locks at Store level.

> Coarsen StoreScanner locks to RegionScanner
> -------------------------------------------
>
>                 Key: HBASE-13082
>                 URL: https://issues.apache.org/jira/browse/HBASE-13082
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>         Attachments: 13082.txt
>
>
> Continuing where HBASE-10015 left of.
> We can avoid locking (and memory fencing) inside StoreScanner by deferring to the lock
already held by the RegionScanner.
> In tests this shows quite a scan improvement and reduced CPU (the fences make the cores
wait for memory fetches).
> There are some drawbacks too:
> * All calls to RegionScanner need to be remain synchronized
> * Implementors of coprocessors need to be diligent in following the locking contract.
For example Phoenix does not lock RegionScanner.nextRaw() and required in the documentation
(not picking on Phoenix, this one is my fault as I told them it's OK)
> * possible starving of flushes and compaction with heavy read load. RegionScanner operations
would keep getting the locks and the flushes/compactions would not be able finalize the set
of files.
> I'll have a patch soon.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message