hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anastasia Braginsky (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14921) Memory optimizations
Date Tue, 19 Apr 2016 12:31:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15247653#comment-15247653
] 

Anastasia Braginsky commented on HBASE-14921:
---------------------------------------------

{quote}
It will be bad, if we have to continue with Cell ref in case of flat map (No extra copy to
new chunks as not much of the cells getting expired/removed). Cells having ref to chunk data
(byte[] now). Can we make the meta data here as ref + offset ( 8 = 4 = 12 bytes per Cell)..Ya
it is 4 bytes more but its ok and better than 40 bytes per cell overhead. We need to mark
the Cells created out of copy to MSLAB in a special way so as to retrieve the byte[] ref.
{quote}

Agree with you that it is better to flatten into the CellChunkMap instead of CellArrayMap.
But as you have said, in order to do that “We need to mark the Cells created out of copy
to MSLAB in a special way so as to retrieve the byte[] ref.”. This is planned to be done
in the future :)

bq. in case of Cells in CSLM, the seqId is a state (long) on Cell object. That will be an
issue in above approach. So when the Cell is recreated after copying to MSLAB (mayCloneWithAllocator),
we can keep append the seqId 8 bytes after Cell key, value, tags.

I am probably not familiar enough, so  I don’t quite understand the problem. If the Cell
is copied to another Chunk in MSLAB, then everything is copied: Cell key, value, tags, seqId,
isn’t it? Or do we care to “waste” those 8 bytes of seqId? Or would it give us wrong
length? [~anoop.hbase], can you please try to explain the problem once again? Thanking you
in advance!

bq. Am I making the explanation here? Any help needed, I can do. :)

Thank you so much again for suggesting your help! If you have extra time, you can deal with
“We need to mark the Cells created out of copy to MSLAB in a special way so as to retrieve
the byte[] ref.” But probably the merging it all together would be a nightmare… So at
least it makes sense to wait to more stable version of this version of the code… What do
you think?

> Memory optimizations
> --------------------
>
>                 Key: HBASE-14921
>                 URL: https://issues.apache.org/jira/browse/HBASE-14921
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: Eshcar Hillel
>            Assignee: Anastasia Braginsky
>         Attachments: CellBlocksSegmentInMemStore.pdf, CellBlocksSegmentinthecontextofMemStore(1).pdf,
HBASE-14921-V01.patch, HBASE-14921-V02.patch, HBASE-14921-V03.patch, IntroductiontoNewFlatandCompactMemStore.pdf
>
>
> Memory optimizations including compressed format representation and offheap allocations



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message