hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16438) Create a cell type so that chunk id is embedded in it
Date Wed, 22 Mar 2017 18:43:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15936900#comment-15936900

Anoop Sam John commented on HBASE-16438:

Not clearing/removing from Q is not the real issue. Code wise there may not be a remove. But
the MSLABImpl object keeps this Q and the MSLAB object become dead once the segment referring
it gets flushed.  So some time we will remove it.  The issue was specific to use case where
there are many updates to same Cell.  MSLAB was in use in that!   So we have cells from one
chunk but all become irrelevant after some time as new cells comes in. Adding new cells to
CSLM would remove the old.  If we were not referring to that chunk any where other than those
original Cells, a GC can collect it.  But adding to the Q was preventing it.   That is why
that jira changed it to keep only chunks from pool.
Now we keep refer to all chunks created out of ChunkCreator. So we are back to old problem

But as such there wont be OOME for sure.  Because as part of above jira [~carp84] fixed another
issue where we account for the cell size (when its data bytes in chunk) and so we will end
up in size breaching at region level/ global level and normal/forced flush will happen.  So
I dont feel like any OOME issue.  But to have a better GC in this scenario, HBASE-16195 would
have helped , which will be broken by this jira.

Am I making things clear?

cc [~carp84]

> Create a cell type so that chunk id is embedded in it
> -----------------------------------------------------
>                 Key: HBASE-16438
>                 URL: https://issues.apache.org/jira/browse/HBASE-16438
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>         Attachments: HBASE-16438_1.patch, HBASE-16438_3_ChunkCreatorwrappingChunkPool.patch,
HBASE-16438_4_ChunkCreatorwrappingChunkPool.patch, HBASE-16438.patch, MemstoreChunkCell_memstoreChunkCreator_oldversion.patch,
> For CellChunkMap we may need a cell such that the chunk out of which it was created,
the id of the chunk be embedded in it so that when doing flattening we can use the chunk id
as a meta data. More details will follow once the initial tasks are completed. 
> Why we need to embed the chunkid in the Cell is described by [~anastas] in this remark
over in parent issue https://issues.apache.org/jira/browse/HBASE-14921?focusedCommentId=15244119&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15244119

This message was sent by Atlassian JIRA

View raw message