hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14921) Memory optimizations
Date Mon, 29 Feb 2016 05:46:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15171434#comment-15171434

stack commented on HBASE-14921:

Really helpful diagram

On #2, you have heard the rumor that MSLAB may not be needed when running on G1GC (TBD)

Should we move the MSLAB forward so when we pull form the socket on the read of requests,
we read into an MSLAB ([~anoop.hbase]?)

So, when you say the MSLAB can be offheap, its ok to have references only in CSLM?  We do
not want to be copying data across the onheap/offheap boundary if it can be avoided. We can
do compare offheap (more a question for our mighty [~anoop.hbase]).

Just to say that we want this compaction-in-memory feature ON by default. Even in the case
where an in-memory compaction is unable to purge any cells because all updates are unique,
this feature should 'cost' no more than our current setup. Lets aim for this. We do not want
you all doing a bunch of work on something that folks have to 'enable'. In such a case, only
the hbase cognescenti will get the benefit of your work. So, it looks like you are talking
of doing at least an extra copy from the original MSLAB to a new Segment MSLAB.  Would be
cool if a bit of evidence that this extra copy to a Segment, even in the worst case where
no purge was possible, cost less than trying to continue with a 'fat' CSLM.

"The compaction process can repeat until we must flush to disk. " There will be guards in
place to prevent our compacting in-memory when it not possible that a compaction can produce
a tighter in-memory representation (no purges possible, etc.)?

First pass. Thanks for the write-up

> Memory optimizations
> --------------------
>                 Key: HBASE-14921
>                 URL: https://issues.apache.org/jira/browse/HBASE-14921
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: Eshcar Hillel
>         Attachments: CellBlocksSegmentInMemStore.pdf
> Memory optimizations including compressed format representation and offheap allocations

This message was sent by Atlassian JIRA

View raw message