hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12282) Ensure Cells and its implementations work with Buffers also
Date Wed, 22 Oct 2014 17:35:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14180217#comment-14180217

Anoop Sam John commented on HBASE-12282:

bq. BBs may have more index and position management internally. BR may come handy here.
What we really need is to have a new impl of BB so that we can avoid overhead in read APIs.
Also the changes can make sure that the inlining will be possible.  So change to BR alone
won't really help us. And it will add extra overhead of BR to BB. Andy had done some testing
for the inlining and we are not getting inlining for some of the APIs.

bq.If we want to deal with BBs one issue in case of DBB is that max size of integer is the
max size of a DBB that can be allocated. How can that be addressed. Can our HFileBlock be
bigger than that?
You mean the ByteBuffer#allocateDirect(int capacity) and int limitation? Any way that is there
with byte[] also no? I think int max size is fair enough

bq.Some of the steps followed in the patch could be hacky but if we could find ways to address
them then we can move towards the goal of BB 
Yes looks hacky. We can think to better it.

We discussed abt having APIs in Cell to do the compares (like row compare) but the issue is
within one Cell impl it knows abt itself but not about the incoming. If we continue to do
this compares based on getxxxArray() APIs, we will end up in many times byte[] creation and
data copy from buffer to this array. This is applicable in the Heap compares in Scanner as
well as some of the Filters (like SCVF)

> Ensure Cells and its implementations work with Buffers also
> -----------------------------------------------------------
>                 Key: HBASE-12282
>                 URL: https://issues.apache.org/jira/browse/HBASE-12282
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver, Scanners
>    Affects Versions: 0.99.1
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0, 0.99.2
>         Attachments: HBASE-12224_2.patch
> This issue can be used to brainstorm and then do the necessary changes for the offheap
work.  All impl of cells deal with byte[] but when we change the Hfileblocks/Readers to work
purely with Buffers then the byte[] usage would mean that always the data is copied to the
onheap.  Cell may need some interface change to implement this.

This message was sent by Atlassian JIRA

View raw message