hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
Date Thu, 27 Oct 2016 08:44:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611207#comment-15611207

Anoop Sam John commented on HBASE-16438:

In CellChunkMap we write the index also to off heap chunks.  So its ok to write the seqId
also along with chunkId+offset+length. So totally 20 bytes per cell but all can be in offheap
area. So we are in better place.
The other way of keeping the ref means we will need a ref array + the chunk. 2 data structures
keep the index data. Per cell one entry to ref array (that is on heap) and remaining 16 bytes
can go to offheap chunk.

Only worry of using chunk from pool for the index is the wastage it can make. May be we should
share one chunk (One BB of 2 MB size) btw more than one segment's index.

Elsewhere [~saint.ack@gmail.com] was mentioning abt upping the chunk size from 2 MB too. 
Then the concern is more. With 2 MB we can keep 131072 cells index data in it.

> Create a cell type so that chunk id is embedded in it
> -----------------------------------------------------
>                 Key: HBASE-16438
>                 URL: https://issues.apache.org/jira/browse/HBASE-16438
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
> For CellChunkMap we may need a cell such that the chunk out of which it was created,
the id of the chunk be embedded in it so that when doing flattening we can use the chunk id
as a meta data. More details will follow once the initial tasks are completed. 
> Why we need to embed the chunkid in the Cell is described by [~anastas] in this remark
over in parent issue https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119

This message was sent by Atlassian JIRA

View raw message