Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 27 Oct 2016 08:44:59 +0000 (UTC)
From: "Anoop Sam John (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12997866.1471428798000.98941.1477557899150@Atlassian.JIRA>
In-Reply-To: <JIRA.12997866.1471428798000@Atlassian.JIRA>
References: <JIRA.12997866.1471428798000@Atlassian.JIRA> <JIRA.12997866.1471428798716@arcas>
Subject: [jira] [Commented] (HBASE-16438) Create a cell type so that chunk
 id is embedded in it
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 27 Oct 2016 08:45:01 -0000


    [ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611207#comment-15611207 ] 

Anoop Sam John commented on HBASE-16438:
----------------------------------------

In CellChunkMap we write the index also to off heap chunks.  So its ok to write the seqId also along with chunkId+offset+length. So totally 20 bytes per cell but all can be in offheap area. So we are in better place.
The other way of keeping the ref means we will need a ref array + the chunk. 2 data structures keep the index data. Per cell one entry to ref array (that is on heap) and remaining 16 bytes can go to offheap chunk.

Only worry of using chunk from pool for the index is the wastage it can make. May be we should share one chunk (One BB of 2 MB size) btw more than one segment's index.

Elsewhere [~saint.ack@gmail.com] was mentioning abt upping the chunk size from 2 MB too.  Then the concern is more. With 2 MB we can keep 131072 cells index data in it.

> Create a cell type so that chunk id is embedded in it
> -----------------------------------------------------
>
>                 Key: HBASE-16438
>                 URL: https://issues.apache.org/jira/browse/HBASE-16438
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>
> For CellChunkMap we may need a cell such that the chunk out of which it was created, the id of the chunk be embedded in it so that when doing flattening we can use the chunk id as a meta data. More details will follow once the initial tasks are completed. 
> Why we need to embed the chunkid in the Cell is described by [~anastas] in this remark over in parent issue https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)