Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Sat, 19 Apr 2014 03:31:18 +0000 (UTC)
From: "Matt Corgan (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12708331.1397499850362.142720.1397878278755@arcas>
In-Reply-To: <JIRA.12708331.1397499850362@arcas>
References: <JIRA.12708331.1397499850362@arcas>
Subject: [jira] [Commented] (HBASE-10974) Improve DBEs read performance by
 avoiding byte array deep copies for key[] and value[]
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974706#comment-13974706 ] 

Matt Corgan commented on HBASE-10974:
-------------------------------------

{quote}One problem faced while checking the performance is that we need to determine a value to determine the key buffer's initial size.{quote}it's probably too late to add this now, but prefix-tree calculates things like this during encoding and stores them in a block header

overall, this seems like a cool strategy to save some copying, but with a complexity cost.  would this be necessary if you can fix the preemptive hfs.next() call?  if so, then seems like it has merit, but if the underlying problem can be fixed then would this problem go away?

> Improve DBEs read performance by avoiding byte array deep copies for key[] and value[]
> --------------------------------------------------------------------------------------
>
>                 Key: HBASE-10974
>                 URL: https://issues.apache.org/jira/browse/HBASE-10974
>             Project: HBase
>          Issue Type: Improvement
>          Components: Scanners
>    Affects Versions: 0.99.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 0.99.0
>
>         Attachments: HBASE-10974_1.patch
>
>
> As part of HBASE-10801, we  tried to reduce the copy of the value [] in forming the KV from the DBEs. 
> The keys required copying and this was restricting us in using Cells and always wanted to copy to be done.
> The idea here is to replace the key byte[] as ByteBuffer and create a consecutive stream of the keys (currently the same byte[] is used and hence the copy).  Use offset and length to track this key bytebuffer.
> The copy of the encoded format to normal Key format is definitely needed and can't be avoided but we could always avoid the deep copy of the bytes to form a KV and thus use cells effectively. Working on a patch, will post it soon.


--
This message was sent by Atlassian JIRA
(v6.2#6252)