hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12295) Prevent block eviction under us if reads are in progress from the BBs
Date Fri, 29 May 2015 03:21:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14564140#comment-14564140

ramkrishna.s.vasudevan commented on HBASE-12295:

bq.You can't figure whether L1 or L2 looking at the key?
You mean add some different type of keys for the L1 or L2 case?  Can check this.
bq.We'll have to dig in on why. You'd think w/ less intermediaries that it would be faster.
Yes that was the reason.  When we were writing every cell to the socket directly things were
really slow.
bq.But pulling the rpc controller back into HRegion is a little perverse.
Okie, we could see if we can make the REsult carry this cellblock.  Will check on this once
bq.(maybe this is not possible....). That said, for now at least, the creation of CellBlock
is a good place for triggering decrement of backing block reference.
Yes. Right.
bq. If that is not enough, can we mimic scan context in Get?
Will see on this if it is really possible. 
bq.So, as is, we'll make a copy per registered CP?
Yes but only if the scanner returned from the CP hook is not implementing the new interface.
But if we allow the REgionScanner itself to use that new interface and the CP tries to use
that new API diligently like how close() would be used, then we are safe we don't need to
do that copy. Anyway for non-java case we need to do that copy.
bq.You want a delayed close, one that completes only after all outstanding scanners are done?
Can we have scanner close set up one of these?
Ya, I tried out something yesterday. Seems to work.  But checking on all the corner cases,
the next patch may have this change.  However I doubt on how a scan next() can be handled.
 It needs some explicit way of decrementing the ref count alone.
But generally the call to close() would be an logical end to the ongoing read process including
decrementing the ref count on the current list of blocks.
bq.So the boolean does what for compacted files? It says, don't evict files that are being
read though they have been compacted? (Is this like the finalizeScan case at all? Where you
want to do a delayed close until no more references?)
A kind of.  But this would happen purely internally. Nothing like explicitly calling from
the compaciton scanners to set this boolean.  Just the block block in the Bucket cache itself
would know this and do it.
bq.One other thought is how you folks thinking of testing this stuff?
Ya as a first measure we have written unit tests to test out these scenarios and ensure we
are doing the ref counting correctly. We are adding more tests to it so that we cover the
basic scenarios.  
For the cluster testing we are planning to reduce the bucket cache size and ensure we have
frequent eviction scenarios and verify whether any of the results are getting corrupted. Any
other suggestion you have here?

> Prevent block eviction under us if reads are in progress from the BBs
> ---------------------------------------------------------------------
>                 Key: HBASE-12295
>                 URL: https://issues.apache.org/jira/browse/HBASE-12295
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver, Scanners
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>         Attachments: HBASE-12295.pdf, HBASE-12295_trunk.patch
> While we try to serve the reads from the BBs directly from the block cache, we need to
ensure that the blocks does not get evicted under us while reading.  This JIRA is to discuss
and implement a strategy for the same.

This message was sent by Atlassian JIRA

View raw message