hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Corgan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10974) Improve DBEs read performance by avoiding byte array deep copies for key[] and value[]
Date Sat, 19 Apr 2014 03:31:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974706#comment-13974706

Matt Corgan commented on HBASE-10974:

{quote}One problem faced while checking the performance is that we need to determine a value
to determine the key buffer's initial size.{quote}it's probably too late to add this now,
but prefix-tree calculates things like this during encoding and stores them in a block header

overall, this seems like a cool strategy to save some copying, but with a complexity cost.
 would this be necessary if you can fix the preemptive hfs.next() call?  if so, then seems
like it has merit, but if the underlying problem can be fixed then would this problem go away?

> Improve DBEs read performance by avoiding byte array deep copies for key[] and value[]
> --------------------------------------------------------------------------------------
>                 Key: HBASE-10974
>                 URL: https://issues.apache.org/jira/browse/HBASE-10974
>             Project: HBase
>          Issue Type: Improvement
>          Components: Scanners
>    Affects Versions: 0.99.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 0.99.0
>         Attachments: HBASE-10974_1.patch
> As part of HBASE-10801, we  tried to reduce the copy of the value [] in forming the KV
from the DBEs. 
> The keys required copying and this was restricting us in using Cells and always wanted
to copy to be done.
> The idea here is to replace the key byte[] as ByteBuffer and create a consecutive stream
of the keys (currently the same byte[] is used and hence the copy).  Use offset and length
to track this key bytebuffer.
> The copy of the encoded format to normal Key format is definitely needed and can't be
avoided but we could always avoid the deep copy of the bytes to form a KV and thus use cells
effectively. Working on a patch, will post it soon.

This message was sent by Atlassian JIRA

View raw message