hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Corgan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10719) Cell interface may need a ByteBuffer rather than a byte[]
Date Wed, 12 Mar 2014 05:45:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931425#comment-13931425

Matt Corgan commented on HBASE-10719:

I debated that when making the Cell interface but thought it might be too big of a step. 
I'm excited to see things are moving to where that's a possibility!  Some thoughts:

* ByteBuffers are heavyweight compared to array
** for HeapBB it would be an additional layer of indirection between Cell and the byte[] that
backs HeapByteBuffer
** looking at source there's more method calls to get to the backing array including big/little-endian
* more complex to work with
* not reusable - have to spawn a new BB with .wrap(..) to move to a new backing array
* encoders copy bytes to assemble the key anyway, so simpler to stick with array for now
* current KeyValue is backed by array, so would be a big step to move everything to BB

* i've seen sometimes DirectBB outperforms array (especially if you don't have to copy it
to byte[] first)
* HeapBB calls probably get inlined enough to match byte[] read performance
* probably better longer term solution if off-heap becomes the norm

I thought a first step could be making only the value field a ByteBuffer (or ByteRange) since
even encoders don't (or shouldn't) need to copy that.

As Nick mentions, maybe ByteRanges with varying implementations are the way to go since they
have a friendlier interface (comparable unsigned bytes).  They could have a ByteBuffer implementation,
albeit with an additional level of indirection over the underlying on-heap byte[] (inlining
to the rescue?).

> Cell interface may need a ByteBuffer rather than a byte[]
> ---------------------------------------------------------
>                 Key: HBASE-10719
>                 URL: https://issues.apache.org/jira/browse/HBASE-10719
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
> See discussion in HBASE-10531. 
> The byte[] backing a cell may be off heap also.  In such a case having a byte[] may require
a copy offheap to on heap.  So better to have ByteBuffer backing a Cell.

This message was sent by Atlassian JIRA

View raw message