hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
Date Thu, 27 Oct 2016 04:56:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15610669#comment-15610669
] 

Anoop Sam John commented on HBASE-16438:
----------------------------------------

Describing diff possibilities at that time.  We keep ref and offset means we can not keep
whole of the index meta in off heap chunk.  chunkid+ offset we can easily keep as both are
ints and we can keep ints in offheap BB.  But the ref means we need to have a ref array which
is on heap.  And we will need another data structure (chunk only) to keep offset + length.
  Yes we can avoid length when there are no tags.  But when tags are there, we need to keep
length :-(. This is because tags parts comes after KL, VL, Key and Value.   Also missed one
more thing at that time of discuss.  ie. we need to keep the seqId 8 bytes).. When cells in
CSLM, it is a long state on the object.  Other way would be that when we copy Cell to MSLAB
(or to a temp byte[]) at the time of addition to Memstore, we need to keep the seqId not as
a state in obj. But that should be put as the last 8 bytes of the cell data bytes. (After
key, value and tags).  When MSLAB is off heap this will help us  to keep more data off heap.
 We need a diff version of cell which can read seqId correctly from last 8 bytes. We will
need decode it.  Will need lots of PoC work around diff ideas.  All these goes to other jira
which says abt ChunkMap variant.

> Create a cell type so that chunk id is embedded in it
> -----------------------------------------------------
>
>                 Key: HBASE-16438
>                 URL: https://issues.apache.org/jira/browse/HBASE-16438
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>
> For CellChunkMap we may need a cell such that the chunk out of which it was created,
the id of the chunk be embedded in it so that when doing flattening we can use the chunk id
as a meta data. More details will follow once the initial tasks are completed. 
> Why we need to embed the chunkid in the Cell is described by [~anastas] in this remark
over in parent issue https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message