hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12295) Prevent block eviction under us if reads are in progress from the BBs
Date Thu, 28 May 2015 20:59:29 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14563680#comment-14563680

stack commented on HBASE-12295:

bq. May be we can only pass the type of the block there? That should be possible. Not a big

You can't figure whether L1 or L2 looking at the key?

bq. When we tried to directly write the cells to the socket as part of the POC things were
directly slow. 

We'll have to dig in on why. You'd think w/ less intermediaries that it would be faster.

bq. Making Result carry it is one option, I think you mean the PB result right?

Ain't sure (smile). I was thinking in both cases. Result would give you a CellScanner... client
or server side.

Result cuts at a different dimension from CellBlock though, right? You get a Result per Get
and per Scan response (could be full row or partial). CellBlock is an RPC primitive. Thought
was that CellBlock could go other places in HBase too to save space (encoding/compression):
e.g. in hfileblock or backing a Result.

bq. The approach here was to be simple use the existing Payload. When you say Result - will
that not be the current way as how we do for non-java clients?


But pulling the rpc controller back into HRegion is a little perverse. HRegion should not
have any rpc pollution. Also, we already pervert rpc controller to carry the payload across
the server/rpc barrier, an unnatural usage. Then, controller itself is not actually doing
any 'controlling' of the rpc -- it is just a place holder in rpc that protobuf actually recommends
we avoid (i.e. generate own rpc interface rather than use the one protoc generates).

bq. Yes, this will be a zero copy. 

But we are copying Cells into CellBlock in memory and then we put the CellBlock on the wire?
Would be nice if we could write the CellBlock directly on the wire (maybe this is not possible....).
That said, for now at least, the creation of CellBlock is a good place for triggering decrement
of backing block reference.

bq. Scans have states and gets do not have states as gets operate with in Region.

Scans are Gets. If that is not enough, can we mimic scan context in Get? Would that help?
 Then we could have one means rather than two keeping account.

bq. ...and the CP tries to use those Cells as its state.

So, as is, we'll make a copy per registered CP?

bq. finalizeScan(boolean finalizeAll).

You want a delayed close, one that completes only after all outstanding scanners are done?
Can we have scanner close set up one of these?

So the boolean does what for compacted files? It says, don't evict files that are being read
though they have been compacted? (Is this like the finalizeScan case at all?  Where you want
to do a delayed close until no more references?)

One other thought is how you folks thinking of testing this stuff?

> Prevent block eviction under us if reads are in progress from the BBs
> ---------------------------------------------------------------------
>                 Key: HBASE-12295
>                 URL: https://issues.apache.org/jira/browse/HBASE-12295
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver, Scanners
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>         Attachments: HBASE-12295.pdf, HBASE-12295_trunk.patch
> While we try to serve the reads from the BBs directly from the block cache, we need to
ensure that the blocks does not get evicted under us while reading.  This JIRA is to discuss
and implement a strategy for the same.

This message was sent by Atlassian JIRA

View raw message