hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-10278) Provide better write predictability
Date Fri, 14 Mar 2014 05:15:54 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13934624#comment-13934624
] 

Jonathan Hsieh edited comment on HBASE-10278 at 3/14/14 5:15 AM:
-----------------------------------------------------------------

The slides help a lot.  Ideally this implementation intent/design would be integrated into
the code similar to the description of how the disruptor patterned wal works.

{quote}
* Added a thread pool which executes the writer.sync() call. SR submits a callable & waits
on the returned Future
{quote}

So one concern is now that the common case has to go through an extra thread context switch.
 Can we make extra context switches only happen on the rarer switching case?

{quote}
* Interrupt SyncRunners to unblock them from current sync call, and wait till all of them
reach a “safepoint”.
* When all SRs are interrupted, they signal switch that they have reached the safepoint, and
Switch process can swap the writer 
* Take all the in-flight edits from all SR(s); append-sync them in same order using the “reserve”
writer
{quote}

So with this approach, we need to wait for interrupts to handle before we can start making
progress using the reserve writer.

Couldn't we just take the roll writer lock, block incoming appends, take the list and start
using the reserve writer when we've initiated the switch process?  We could avoid the extra
thread context switches in the common case.  We may end up with potential duplicate edits
in the old log and new reserve log but that is ok.  instead of interrupting the new sync thread
pool threads, we'd interrupt the syncrunner threads after we've moved to the reserve writer
potentially after we have unblocked the ring buffer handler and even after the roll writer
lock.


was (Author: jmhsieh):
{quote}
* Added a thread pool which executes the writer.sync() call. SR submits a callable & waits
on the returned Future
{quote}

So one concern is now that the common case has to go through an extra thread context switch.
 Can we make extra context switches only happen on the rarer switching case?

{quote}
* Interrupt SyncRunners to unblock them from current sync call, and wait till all of them
reach a “safepoint”.
* When all SRs are interrupted, they signal switch that they have reached the safepoint, and
Switch process can swap the writer 
* Take all the in-flight edits from all SR(s); append-sync them in same order using the “reserve”
writer
{quote}

So with this approach, we need to wait for interrupts to handle before we can start making
progress using the reserve writer.

Couldn't we just take the roll writer lock, block incoming appends, take the list and start
using the reserve writer when we've initiated the switch process?  We could avoid the extra
thread context switches in the common case.  We may end up with potential duplicate edits
in the old log and new reserve log but that is ok.  instead of interrupting the new sync thread
pool threads, we'd interrupt the syncrunner threads after we've moved to the reserve writer
potentially after we have unblocked the ring buffer handler and even after the roll writer
lock.

> Provide better write predictability
> -----------------------------------
>
>                 Key: HBASE-10278
>                 URL: https://issues.apache.org/jira/browse/HBASE-10278
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Himanshu Vashishtha
>            Assignee: Himanshu Vashishtha
>         Attachments: 10278-wip-1.1.patch, Multiwaldesigndoc.pdf, SwitchWriterFlow.pptx
>
>
> Currently, HBase has one WAL per region server. 
> Whenever there is any latency in the write pipeline (due to whatever reasons such as
n/w blip, a node in the pipeline having a bad disk, etc), the overall write latency suffers.

> Jonathan Hsieh and I analyzed various approaches to tackle this issue. We also looked
at HBASE-5699, which talks about adding concurrent multi WALs. Along with performance numbers,
we also focussed on design simplicity, minimum impact on MTTR & Replication, and compatibility
with 0.96 and 0.98. Considering all these parameters, we propose a new HLog implementation
with WAL Switching functionality.
> Please find attached the design doc for the same. It introduces the WAL Switching feature,
and experiments/results of a prototype implementation, showing the benefits of this feature.
> The second goal of this work is to serve as a building block for concurrent multiple
WALs feature.
> Please review the doc.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message