hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
Date Mon, 19 Oct 2015 15:57:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14963502#comment-14963502
] 

ramkrishna.s.vasudevan commented on HBASE-13082:
------------------------------------------------

bq.What about flush? Once the flush is finished, we will have to clear the snapshot of cells
in Memstore no? Then we will have to open this newly added file for reading.. If we dont open
it, we can not release the memstore snapshot. 
We do open the reader for sure. But one thing may be to check is that if we open but still
the current scanner heap is not reset will it have any impact on the current scan is what
needs to be checked?  Because during flush the reads can still be served from the snapshot.
Only after flush is a point to be noted. 
bq.This status is in reader? It suits better in StoreFile no?
The reader is the common object here.  Hence was referencing it from there. Initially had
it in storefile only but felt that StoreFile is more volatile. Let me check. Can be done.
 Infact I was also not very sure of having the state in the reader. 
We can change the states no problem.
bq.This is a new Chore thread adding to the system.. Mention about it clearly. It will be
one thread per RS. Also what is its interval? Is it configurable? How?
For to add that config details. It is per store and the chore is started by the HStore and
not the RS. 
bq.Hence it requires us to add new APIs which ensures that a split can go ahead even if reference
files are present but they are in the COMPACTED state
This is bit of sticky area.  In case of merge I have tried to forcefully clear the compacted
files. May be we need to do the same with split also. But in split do we explicitly call compact?
 I was not pretty sure on that. But in merge we do. 



> Coarsen StoreScanner locks to RegionScanner
> -------------------------------------------
>
>                 Key: HBASE-13082
>                 URL: https://issues.apache.org/jira/browse/HBASE-13082
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>            Assignee: ramkrishna.s.vasudevan
>         Attachments: 13082-test.txt, 13082-v2.txt, 13082-v3.txt, 13082-v4.txt, 13082.txt,
13082.txt, HBASE-13082.pdf, HBASE-13082_1_WIP.patch, HBASE-13082_2_WIP.patch, HBASE-13082_3.patch,
HBASE-13082_4.patch, gc.png, gc.png, gc.png, hits.png, next.png, next.png
>
>
> Continuing where HBASE-10015 left of.
> We can avoid locking (and memory fencing) inside StoreScanner by deferring to the lock
already held by the RegionScanner.
> In tests this shows quite a scan improvement and reduced CPU (the fences make the cores
wait for memory fetches).
> There are some drawbacks too:
> * All calls to RegionScanner need to be remain synchronized
> * Implementors of coprocessors need to be diligent in following the locking contract.
For example Phoenix does not lock RegionScanner.nextRaw() and required in the documentation
(not picking on Phoenix, this one is my fault as I told them it's OK)
> * possible starving of flushes and compaction with heavy read load. RegionScanner operations
would keep getting the locks and the flushes/compactions would not be able finalize the set
of files.
> I'll have a patch soon.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message