hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eshcar Hillel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline
Date Mon, 09 Jan 2017 11:51:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15811590#comment-15811590

Eshcar Hillel commented on HBASE-17434:

The logic and timing of moving segments into the pipeline and out of it is done by CompactingMemStore.
CompactingMemStore maintains two invariants/conditions:
(1) not to remove a segment before it is added to the next entity in the active-pipeline-snapshot
(2) when creating the scanners for the memstore create them in the same order as the data
flows, namely active then pipeline then snapshot.
With these two invariants we are guaranteed not to skip any data in a get or scan operation
but may scan the same segment twice.

If and when getScanners() is added to CompactionPipeline then it needs to follow the guidelines
in the documentation: get a local copy of the readOnlyCopy and create scanners for the local

> New Synchronization Scheme for Compaction Pipeline
> --------------------------------------------------
>                 Key: HBASE-17434
>                 URL: https://issues.apache.org/jira/browse/HBASE-17434
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Eshcar Hillel
>            Assignee: Eshcar Hillel
>         Attachments: HBASE-17434-V01.patch, HBASE-17434-V02.patch, HBASE-17434-V03.patch
> A new copyOnWrite synchronization scheme is introduced for the compaction pipeline.
> The new scheme is better since it removes the lock from getSegments() which is invoked
in every get and scan operation, and it reduces the number of LinkedList objects that are
created at runtime, thus can reduce GC (not by much, but still...).
> In addition, it fixes the method getTailSize() in compaction pipeline. This method creates
a MemstoreSize object which comprises the data size and the overhead size of the segment and
needs to be atomic.

This message was sent by Atlassian JIRA

View raw message