Return-Path: <issues-return-254715-archive-asf-public=cust-asf.ponee.io@hbase.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id 64218200B59
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Mon,  8 Aug 2016 20:36:27 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id 629D9160AB3; Mon,  8 Aug 2016 18:36:27 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id A9BF4160A77
	for <archive-asf-public@cust-asf.ponee.io>; Mon,  8 Aug 2016 20:36:26 +0200 (CEST)
Received: (qmail 50995 invoked by uid 500); 8 Aug 2016 18:36:20 -0000
Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:issues-help@hbase.apache.org>
List-Unsubscribe: <mailto:issues-unsubscribe@hbase.apache.org>
List-Post: <mailto:issues@hbase.apache.org>
List-Id: <issues.hbase.apache.org>
Delivered-To: mailing list issues@hbase.apache.org
Received: (qmail 50967 invoked by uid 99); 8 Aug 2016 18:36:20 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 Aug 2016 18:36:20 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 7D71C2C029E
	for <issues@hbase.apache.org>; Mon,  8 Aug 2016 18:36:20 +0000 (UTC)
Date: Mon, 8 Aug 2016 18:36:20 +0000 (UTC)
From: "Anoop Sam John (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12954189.1459226782000.251655.1470681380511@Atlassian.JIRA>
In-Reply-To: <JIRA.12954189.1459226782000@Atlassian.JIRA>
References: <JIRA.12954189.1459226782000@Atlassian.JIRA> <JIRA.12954189.1459226782861@arcas>
Subject: [jira] [Commented] (HBASE-15554)
 StoreFile$Writer.appendGeneralBloomFilter generates extra KV
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
archived-at: Mon, 08 Aug 2016 18:36:27 -0000


    [ https://issues.apache.org/jira/browse/HBASE-15554?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D154=
12234#comment-15412234 ]=20

Anoop Sam John commented on HBASE-15554:
----------------------------------------

Am sorry if I was not saying it clear. I dont mean still patch is having du=
plicate. What I mean is when I say Iterator based HashKey, I wanted it to b=
e single structure we use with Hash rather than byte[]/BB/Cell..  But if th=
e algo demands an offset based byte getter am fine.
bq.one is that what ever be the cell format we should finally assume the ba=
ck end is KV format key only. Because the offset and length that we pass to=
 the hash algo is assuming that it is continuous
Why we need pass an offset to hash() function?  We need pass HashKey. Inter=
nally the impl of HashKey has to know which byte to be returned when getter=
s are called on it. Ya if u dont have iterator model u will have get(int) w=
hich return byte.  So the Hash functions has to call get() based on relativ=
e offset eg: get(0), get(1) etc.  Not like cur way of offset+1, offset+2.  =
 When the impl gets these calls, it has to convert it into absolute offsets=
.  It is not that simple in ROW_COL case.  Here based on the coming offset =
you have it map it which area of the Cell this belongs also. That is what I=
 was trying to say.  When get(0) or get(1) is called, those comes in rkLen =
part.  get(2) -  get(<rkLen>+2)   these belong to rk bytes.    So will have=
 to deal some sort of math.  So you really dont have to assume that the Cel=
l is of KV serialization.   Just like in the past which all bytes of Cell w=
here , continue to use those.  Am I making it clear now?   It would be good=
 if we can remove any sort of KV assumption from the code path.  I think it=
 is pending only in this Bloom area.

> StoreFile$Writer.appendGeneralBloomFilter generates extra KV
> ------------------------------------------------------------
>
>                 Key: HBASE-15554
>                 URL: https://issues.apache.org/jira/browse/HBASE-15554
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Performance
>            Reporter: Vladimir Rodionov
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15554.patch, HBASE-15554_10.patch, HBASE-15554=
_3.patch, HBASE-15554_4.patch, HBASE-15554_6.patch, HBASE-15554_7.patch, HB=
ASE-15554_9.patch
>
>
> Accounts for 10% memory allocation in compaction thread when BloomFilterT=
ype is ROWCOL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)