hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
Date Wed, 26 Oct 2016 05:35:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15607499#comment-15607499

ramkrishna.s.vasudevan commented on HBASE-16438:

Thanks for your comments. The case I think of is this.
We do in memory flushes now. So every in memory flush will create a segment.
The cells are located in this segment. When MSLAB is ON and chunk pool is ON we create a chunk
from the pool and that is where the cells are added. 
Now this segment is of type CSLM each entry pointing to the chunk created from the chunk pool.

When I try to create an index out of this I don't add the chunk where the cell is added. Instead
I just add the chunkid (assume when we ask for a chunk from the pool it gives out an id) and
the cell's offset and length.
Why chunk id is important is because again say there is one more in memory flush and we create
one more segment and the old one is pushed to the pipeline. So we convert this segment from
CSLM to chunk based. Now in my index array since I have another segment added I  need to clearly
know for which segment was this chunk created because the cell's offset and length could be
same and there is no unique way to distinguish.
In the CellArrayMap impl there is no such need because just that they create an array of cells
and every index in the array points to the cell referenced inside the chunks. We don't need
any chunkid there. Because it points to actual cells. But in CellChunkIndex we don't point
to actual cells.
Let me know if I am writing it clearly if not can rephrase again. Thanks.

> Create a cell type so that chunk id is embedded in it
> -----------------------------------------------------
>                 Key: HBASE-16438
>                 URL: https://issues.apache.org/jira/browse/HBASE-16438
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
> For CellChunkMap we may need a cell such that the chunk out of which it was created,
the id of the chunk be embedded in it so that when doing flattening we can use the chunk id
as a meta data. More details will follow once the initial tasks are completed. 

This message was sent by Atlassian JIRA

View raw message