hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anastasia Braginsky (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17434) New Synchronization Scheme for Compaction Pipeline
Date Mon, 09 Jan 2017 12:50:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15811698#comment-15811698

Anastasia Braginsky commented on HBASE-17434:

bq. Without HBASE-17081 in play, we wouldn't know for sure that the proposed change would
fix HBASE-17379.

I just want to clarify some facts:
1. A bug in HBASE-17081 was an access to the compaction pipeline for the getScanners() method
without synchronization.
2. HBASE-17379 was opened when this issue was found.
3. A bug in HBASE-17081 could be fixed, but instead its commit was reverted.
4. While investigating the problem we saw that there was a problem with the synchronization
of pipeline's getSize() even before HBASE-17081.
5. Then HBASE-17434 was opened to totally change the synchronization scheme of the compaction
pipeline and to make it more efficient (meaning to allow more parallel executions in the same

So regarding, whether or not we can be sure that the proposed change can fix HBASE-17379,
this is not related to HBASE-17081, but to the problem it exposed.
We should prove there is no concurrent changes of the pipeline content that can be seen inconsistently
by different threads. If this is the case, all problems are fixed.

> New Synchronization Scheme for Compaction Pipeline
> --------------------------------------------------
>                 Key: HBASE-17434
>                 URL: https://issues.apache.org/jira/browse/HBASE-17434
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Eshcar Hillel
>            Assignee: Eshcar Hillel
>         Attachments: HBASE-17434-V01.patch, HBASE-17434-V02.patch, HBASE-17434-V03.patch
> A new copyOnWrite synchronization scheme is introduced for the compaction pipeline.
> The new scheme is better since it removes the lock from getSegments() which is invoked
in every get and scan operation, and it reduces the number of LinkedList objects that are
created at runtime, thus can reduce GC (not by much, but still...).
> In addition, it fixes the method getTailSize() in compaction pipeline. This method creates
a MemstoreSize object which comprises the data size and the overhead size of the segment and
needs to be atomic.

This message was sent by Atlassian JIRA

View raw message