Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 28 Apr 2015 06:03:06 +0000 (UTC)
From: "Anoop Sam John (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12708331.1397499850000.3730.1430200986626@Atlassian.JIRA>
In-Reply-To: <JIRA.12708331.1397499850000@Atlassian.JIRA>
References: <JIRA.12708331.1397499850000@Atlassian.JIRA>
 <JIRA.12708331.1397499850362@arcas>
Subject: [jira] [Commented] (HBASE-10974) Improve DBEs read performance by
 avoiding byte array deep copies for key[] and value[]
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516446#comment-14516446 ] 

Anoop Sam John commented on HBASE-10974:
----------------------------------------

bq.if we really know how much was the exact bytes that was encoded we could create those many BBs offheap,
You mean create one BB of that much size I believe.   Then that will be too much of memory consumption no? You are doubling (in fact more that double) the memory need for the scan.  I think that will be a problem.

> Improve DBEs read performance by avoiding byte array deep copies for key[] and value[]
> --------------------------------------------------------------------------------------
>
>                 Key: HBASE-10974
>                 URL: https://issues.apache.org/jira/browse/HBASE-10974
>             Project: HBase
>          Issue Type: Improvement
>          Components: Scanners
>    Affects Versions: 0.99.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0, 1.2.0
>
>         Attachments: HBASE-10974_1.patch
>
>
> As part of HBASE-10801, we  tried to reduce the copy of the value [] in forming the KV from the DBEs. 
> The keys required copying and this was restricting us in using Cells and always wanted to copy to be done.
> The idea here is to replace the key byte[] as ByteBuffer and create a consecutive stream of the keys (currently the same byte[] is used and hence the copy).  Use offset and length to track this key bytebuffer.
> The copy of the encoded format to normal Key format is definitely needed and can't be avoided but we could always avoid the deep copy of the bytes to form a KV and thus use cells effectively. Working on a patch, will post it soon.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)