hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anastasia Braginsky (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
Date Mon, 27 Mar 2017 11:44:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15943108#comment-15943108

Anastasia Braginsky commented on HBASE-16438:

bq. But am not sure how you mean this hardening it again. 

public class SoftReference<T> extends Reference<T>
So I think you can use just reference in the map type and later just to update the map with
new mapping (delete old mapping from some chunkID to soft reference and add a new mapping
from same chunkID to hard reference). If this doesn't work for some reason we may have two
maps <Long, SoftRef(Chunk)> and <Long, HardRef(Chunk)>.

bq. Now when a segment is getting converted from CSLM based into CellChunkMap based, can we
just track the chunkIds actually getting used? (Which cells getting moved into ChunkMap).
All the remaining chunks seems of no use and can be immediate removed from the map.

Nice idea. I think the result of this idea is the same as combining soft and hard pointers
in the Creator's Map. However, with combining soft and hard pointers in the Creator's Map
the chunks can be GCed also in active segment.

bq. Do you feel that can work? I have implemented the same in a patch but not posted it as
I need to see if there is really a perf penalty.

I saw this idea, I feel it can work, I think it is a bit complicated and may cost us some
performance. I believe we can achieve the same for less. But if you want to go for it, I am
OK with that.

> Create a cell type so that chunk id is embedded in it
> -----------------------------------------------------
>                 Key: HBASE-16438
>                 URL: https://issues.apache.org/jira/browse/HBASE-16438
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>         Attachments: HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch,
HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch,
> For CellChunkMap we may need a cell such that the chunk out of which it was created,
the id of the chunk be embedded in it so that when doing flattening we can use the chunk id
as a meta data. More details will follow once the initial tasks are completed. 
> Why we need to embed the chunkid in the Cell is described by [~anastas] in this remark
over in parent issue https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119

This message was sent by Atlassian JIRA

View raw message