hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
Date Mon, 27 Mar 2017 10:51:42 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15943037#comment-15943037
] 

Anoop Sam John commented on HBASE-16438:
----------------------------------------

I can think of some way. Not sure whether it is crazy or not.
We have the id vs chunk Mapping globally in ChunkCreator.   Also we will have chunk ids tracked
in MSLABImpl which this mslab deals with.  Now when a segment is getting converted from CSLM
based into CellChunkMap based,  can we just track the chunkIds actually getting used? (Which
cells getting moved into ChunkMap). All the remaining chunks seems of no use and can be immediate
removed from the map.   Still while the segment is active those chunks can not get GCed, which
is not the case right now..  But once the in memory flush happens, there can get GCed.   Just
throwing out some thoughts.

Any way as said, there is no possibility any OOME which was fixed by Yu Li.   Only thing is
better GC possibility we are lossing.

> Create a cell type so that chunk id is embedded in it
> -----------------------------------------------------
>
>                 Key: HBASE-16438
>                 URL: https://issues.apache.org/jira/browse/HBASE-16438
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>         Attachments: HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch,
HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch,
MemstoreChunkCell_trunk.patch
>
>
> For CellChunkMap we may need a cell such that the chunk out of which it was created,
the id of the chunk be embedded in it so that when doing flattening we can use the chunk id
as a meta data. More details will follow once the initial tasks are completed. 
> Why we need to embed the chunkid in the Cell is described by [~anastas] in this remark
over in parent issue https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message