hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anastasia Braginsky (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17662) Disable in-memory flush when replaying from WAL
Date Sun, 26 Feb 2017 10:48:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884714#comment-15884714

Anastasia Braginsky commented on HBASE-17662:

Hey everyone! Here are my answers:


bq. Will the thread that sets the state be same as the one reading it?

Yes, the thread that sets the 'inWalReplay' flag is the replay thread, which is the same thread
that performs the adds and thus the same thread that is going to try to flush in memory and
then read the 'inWalReplay' flag when the memstore size grows above the threshold.

bq. Is this what single-threaded presumption around wal replay means? 

No, the single-threaded presumption is that there are no two (or more) replays simultaneously
on the same store.

bq. If single-threaded why are there concerns around in-memory flush? It only works if update
lock taken? 

Because in-memory flush is done one a separate thread (T), which is dispatched once the memstore
size grows above the threshold. This thread T takes the update lock in order to protect movement
of the active segment to pipeline and creating the new active segment. Threre should be no
concurrent adds to the memstore in the same time. Thread T assumes the adds are taking the
update lock in shared mode and thus when T takes it in exclusive mode there are no concurrent
updates. However, this assumption doesn't hold in replay case, where memstore is updated without
taking this lock. From here, T takes the lock successfully and moves the active segment under
the arms of ongoing concurrent updates. This is the concern.

bq. (Flag can't be volatile; that'd be too expensive if we have to check it on each update
to memstore).

Flag is not volatile and I changed it to be simple boolean and not atomic boolean. The flag
is checked only when the memstore size grows above the threshold. Therefore in common code
path, it only will be checked once in a while.

[~anoop.hbase] and [~ram_krish], thanks for your comments. As I said above, I've changed 'inWalReplay'
flag to be simple boolean and it is checked only under the 'if' condition that checks for
the size.

> Disable in-memory flush when replaying from WAL
> -----------------------------------------------
>                 Key: HBASE-17662
>                 URL: https://issues.apache.org/jira/browse/HBASE-17662
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Anastasia Braginsky
>            Assignee: Anastasia Braginsky
>         Attachments: HBASE-17662-V02.patch, HBASE-17662-V03.patch, HBASE-17662-V04.patch,
> When replaying the edits from WAL, the region's updateLock is not taken, because a single
threaded action is assumed. However, the thread-safeness of the in-memory flush of CompactingMemStore
is based on taking the region's updateLock. 
> The in-memory flush can be skipped in the replay time (anyway everything is flushed to
disk just after the replay). Therefore it is acceptable to just skip the in-memory flush action
while the updates come as part of replay from WAL.

This message was sent by Atlassian JIRA

View raw message