hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eshcar Hillel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14920) Compacting Memstore
Date Sun, 22 May 2016 14:08:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295576#comment-15295576

Eshcar Hillel commented on HBASE-14920:

Thank you [~ram_krish] [~anoop.hbase] and [~stack] for your feedback, the feature would not
have been in such a great shape without your helpful comments.

Answering your questions:
bq. Just don't do this if family.isInMemoryCompaction() is true. Just declare className and
move the conf.get to the else part.
Aggree. Already asked Anastasia to fix this as part of HBASE-14921.

bq. 'DEEP_OVERHEAD_PER_PIPELINE_ITEM' - Always has the TIMERANGE also. Only ImmutableSegment
has it right?
Right. Pipeline items are always immutable segments so this is ok.

bq. So active gets added, then the segments in the pipleline and then the snapshot. That order+1
is needed?
‘order’ is initially the size of the pipeline. For example, if the pipeline has 2 segments,
active segment sets order=3, pipeline recent segment sets order=2, pipeline old segment sets
order=1, and snapshot sets order=0.

bq. Once again getsnapshot() will add the tail element? Pls check.
getSnapshot() simply retrieve the reference of the snapshot segment.

bq. How does close() region handle the complete flushing?
The method HRegion::doClose() was updated to continue flushing as long as the size of the
region decreases (+some fixed number for potential failures in flush). This is a reasonable
solution for now as the pipeline only has 1 or 2 segments. If this is changed in future Jiras,
this solution can be revised.

bq. Coming to a normal case using CompactingMemstore where we don't have much duplicates as
in your use case, how effective is that WAL.updateStore() is going to be? Can that be avoided
for a case where there are not much compaction happening (i mean no duplicates)?
One option is not to have compacting memstore in this case. Second option will be considered
as part of HBASE-14921; to apply compaction only when there is potential gain. Finally updating
the store’s wal sequence number is not that much to pay every once in awhile.

> Compacting Memstore
> -------------------
>                 Key: HBASE-14920
>                 URL: https://issues.apache.org/jira/browse/HBASE-14920
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Eshcar Hillel
>            Assignee: Eshcar Hillel
>         Attachments: HBASE-14920-V01.patch, HBASE-14920-V02.patch, HBASE-14920-V03.patch,
HBASE-14920-V04.patch, HBASE-14920-V05.patch, HBASE-14920-V06.patch, HBASE-14920-V07.patch,
HBASE-14920-V08.patch, HBASE-14920-V09.patch, HBASE-14920-V10.patch, move.to.junit4.patch
> Implementation of a new compacting memstore with non-optimized immutable segment representation

This message was sent by Atlassian JIRA

View raw message