hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14920) Compacting Memstore
Date Thu, 19 May 2016 12:07:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290994#comment-15290994

ramkrishna.s.vasudevan commented on HBASE-14920:

Thanks for the great work. I have just few questions and some small comments. I won't any
way block the commit and always we can do some work after testing. We also plan to test this
patch along with HBASE-14921 also.

Minor comments from code
String className = conf.get(MEMSTORE_CLASS_NAME, DefaultMemStore.class.getName());
Just don't do this if family.isInMemoryCompaction() is true. Just declare className and move
the conf.get to the else part. 
'DEEP_OVERHEAD_PER_PIPELINE_ITEM' - Always has the TIMERANGE also.  Only ImmutableSegment
has it right?  Still it is needed?  

LinkedList<SegmentScanner> list = new LinkedList<SegmentScanner>();
    list.add(getActive().getSegmentScanner(readPt, order+1));
    for (Segment item : pipelineList) {
      list.add(item.getSegmentScanner(readPt, order));
    list.add(getSnapshot().getSegmentScanner(readPt, order));
So active gets added, then the segments in the pipleline and then the snapshot. That order+1
is needed?

Also since we add the segments from the pipeline also - is the tail element already added?
 Once again getsnapshot() will add the tail element? Pls check. I may be wrong here.
I read through the other comments in this JIRA. 
How does close() region handle the complete flushing?  I saw that there was some suggestion
to pass a boolean indicating to flush all..Is that being done now? Not sure how Region.close()
gets handled.  So is it like the last in memory compaction will get over and by the time we
will have only one segement and that will get flushed as part of the existing code only? 
If it is so then fine.
Coming to a normal case using CompactingMemstore where we don't have much duplicates as in
your use case, how effective is that WAL.updateStore() is going to be?  Can that be avoided
for a case where there are not much compaction happening (i mean no duplicates)?
 I agree that documenting what this CompactingMemstore does is every important and when HBASE-14921
is also combined we may really get the actual benefit by avoiding the memory overhead. 
Thanks for the great work. 

> Compacting Memstore
> -------------------
>                 Key: HBASE-14920
>                 URL: https://issues.apache.org/jira/browse/HBASE-14920
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Eshcar Hillel
>            Assignee: Eshcar Hillel
>         Attachments: HBASE-14920-V01.patch, HBASE-14920-V02.patch, HBASE-14920-V03.patch,
HBASE-14920-V04.patch, HBASE-14920-V05.patch, HBASE-14920-V06.patch, HBASE-14920-V07.patch,
HBASE-14920-V08.patch, HBASE-14920-V09.patch, HBASE-14920-V10.patch, move.to.junit4.patch
> Implementation of a new compacting memstore with non-optimized immutable segment representation

This message was sent by Atlassian JIRA

View raw message