hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anastasia Braginsky (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
Date Thu, 06 Apr 2017 13:33:42 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15958938#comment-15958938

Anastasia Braginsky commented on HBASE-16438:

Hi Guys!

I really apologize for this late comment. If you haven't yet committed, please consider this
I just had time to went over the last version of the patch once again. I left some comment
in the RB, please take a look there, but most of them are not very important. 
The very important thing is that I have just realized that seqID is not written anywhere in
the ByteBuffer! Please correct me if I am wrong...

I was sure (that even before this patch) the serialization of the Cell included writing of
the seqID to the ByteBuffer and we are talking only about CellChunkMap. Obviously, if seqID
is not written in MSLAB Chunk upon cell creation (when the cell is still indexed with CSLM)
it can not be added there later when we have the transfer to CellChunkMap. So the only two
options are (1) to add seqID in the index-chunk when CellChunkMap is created (in-memory-flush)
or (2) to write seqID into the data-chunk when a cell is added to the memstore IN ADDITION
to seqID continue existing in the Cell Object.

I strongly suggest to do the second option.
First, if seqID remains in the Cell object, there is completely no new work being done in
the DefaultMemStore or active segment. The seqID in the data chunk are not going to be accessed
by anyone except CellChunkMap.
Second, addition of another 8 bytes to key+value, which is anyway big sized is not significant.
While adding those 8 bytes to cell-representation in the ChunkMap is increasing the metadata
per cell almost twice.

I really appreciate all the hard work that you are doing! You were talking about this issue
even before, I am sorry I didn't understand that. But I think this task is not complete before
we finalize this seqID issue...

> Create a cell type so that chunk id is embedded in it
> -----------------------------------------------------
>                 Key: HBASE-16438
>                 URL: https://issues.apache.org/jira/browse/HBASE-16438
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>         Attachments: HBASE-16438_10_ChunkCreatorwrappingChunkPool_withchunkRef.patch,
HBASE-16438_11_ChunkCreatorwrappingChunkPool_withchunkRef.patch, HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch,
HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, HBASE-16438_8_ChunkCreatorwrappingChunkPool_withchunkRef.patch,
HBASE-16438_9_ChunkCreatorwrappingChunkPool_withchunkRef.patch, HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch,
> For CellChunkMap we may need a cell such that the chunk out of which it was created,
the id of the chunk be embedded in it so that when doing flattening we can use the chunk id
as a meta data. More details will follow once the initial tasks are completed. 
> Why we need to embed the chunkid in the Cell is described by [~anastas] in this remark
over in parent issue https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119

This message was sent by Atlassian JIRA

View raw message