hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11425) Cell/DBB end-to-end on the read-path
Date Wed, 11 Mar 2015 08:02:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356459#comment-14356459
] 

Anoop Sam John commented on HBASE-11425:
----------------------------------------

You say about this case hbase.block.data.cachecompressed = true right?   Yes this recently
added feature allow to keep block data in compressed form in BC. When this block is read from
BC, these happens
 Step 1 : We create a new on heap buffer and copy compressed data from buckets into it. Make
an HFileBlock backed by this compressed data
 Step 2 : Unpack this block. Then we will create a new byte[] with size equal to the uncompresed
data size for this block. The Compress algo will do uncompress of the block data into this
new buffer

As in our changes we avoid this step 1 new buffer and copy need. We create a block backed
by MBB. We have new InputStream over MBB and pass that for uncomress.  Yes Step 2 will be
still there.
So adv here also. Make some sense?

> Cell/DBB end-to-end on the read-path
> ------------------------------------
>
>                 Key: HBASE-11425
>                 URL: https://issues.apache.org/jira/browse/HBASE-11425
>             Project: HBase
>          Issue Type: Umbrella
>          Components: regionserver, Scanners
>    Affects Versions: 0.99.0
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>         Attachments: Offheap reads in HBase using BBs_final.pdf
>
>
> Umbrella jira to make sure we can have blocks cached in offheap backed cache. In the
entire read path, we can refer to this offheap buffer and avoid onheap copying.
> The high level items I can identify as of now are
> 1. Avoid the array() call on BB in read path.. (This is there in many classes. We can
handle class by class)
> 2. Support Buffer based getter APIs in cell.  In read path we will create a new Cell
with backed by BB. Will need in CellComparator, Filter (like SCVF), CPs etc.
> 3. Avoid KeyValue.ensureKeyValue() calls in read path - This make byte copy.
> 4. Remove all CP hooks (which are already deprecated) which deal with KVs.  (In read
path)
> Will add subtasks under this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message