hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anastasia Braginsky (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
Date Mon, 27 Mar 2017 09:43:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15942943#comment-15942943

Anastasia Braginsky commented on HBASE-16438:

bq. I dont think Soft ref can be used. Once the CSLM is converted to CellChunkMap, we dont
have any hard refs to Chunks from any where (Cells are gone). Now the ChunkMap contain chunkId
and offset , length info. Now assume a GC picked and removed these chunks (Ya there are no
hard refs to it), and after a read comes and we want to read back that Cell, what to do? !!!!

We can "harden" the references in the map in the process of transferring to CellChunkMap (flattening).
The chunks that can not be reached will not be a part of this process and their IDs won't
be found so their references are not going to be "harden", which is OK.

bq. So the Chunk that is put in the map will have a soft reference?

Yes. The chunk that is used for CSLM or CellArrayMap is going to have soft reference the chunk
that is used for CellChunkMap is going to have hard reference.

bq. Can we say CellChunkMap to be used along with off heap MSLAB pool usage only?

This is the first question to answer. I think we should not limit the CellChunkMap to off-heap
right now. As we have seen with CompactingMemStore only when you implement it all and make
a good performance testing you can see what is benefit. As now we see the merge is a good
thing to do, although in the past we already thought to remove this code. So if we implement
CellChunkMap now without any possibility to use it on-heap, we are limiting ourselves without
any experimental evidence. I believe we should write the code generally enough.

> Create a cell type so that chunk id is embedded in it
> -----------------------------------------------------
>                 Key: HBASE-16438
>                 URL: https://issues.apache.org/jira/browse/HBASE-16438
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>         Attachments: HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch,
HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch,
> For CellChunkMap we may need a cell such that the chunk out of which it was created,
the id of the chunk be embedded in it so that when doing flattening we can use the chunk id
as a meta data. More details will follow once the initial tasks are completed. 
> Why we need to embed the chunkid in the Cell is described by [~anastas] in this remark
over in parent issue https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119

This message was sent by Atlassian JIRA

View raw message