hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14921) Memory optimizations
Date Thu, 24 Mar 2016 05:53:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209811#comment-15209811

stack commented on HBASE-14921:

bq. The suggested structure uses 4 times less memory than the previous ConcurrentSkipListMap,
provides no fragmentation and it is ready for off-heaping.


bq. Thanks for taking a look. It looks like the name “CellBlock" made a confusion.

Yeah, needs another name... or maybe better to say, there is no defined 'CellBlock' currently
-- which is a problem to be addressed -- but if we defined it, it would not be what is here...
an implementation of a Map; it would be more basic (A CellScanner only?).

I went back to the nice doc.

bq. The reference is just a machine word and thus occupies less memory than the Cell Object.

What is the 'machine word' and how does it preserve sort?

I need to read this doc a few times...

On the patch..

Its fun. I like what is going on in here.

CellBlockObjectArray presumes passed in array of Cells already sorted? 

Calls to createCellBlocks are only when making subMaps? Who else would call them?

Let me give more review tomorrow. This is interesting...

> Memory optimizations
> --------------------
>                 Key: HBASE-14921
>                 URL: https://issues.apache.org/jira/browse/HBASE-14921
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: Eshcar Hillel
>            Assignee: Anastasia Braginsky
>         Attachments: CellBlocksSegmentInMemStore.pdf, CellBlocksSegmentinthecontextofMemStore(1).pdf,
HBASE-14921-V01.patch, HBASE-14921-V02.patch
> Memory optimizations including compressed format representation and offheap allocations

This message was sent by Atlassian JIRA

View raw message