hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10278) Provide better write predictability
Date Tue, 16 Dec 2014 23:55:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14249181#comment-14249181

stack commented on HBASE-10278:

[~busbey] Could do the 0.89fb tack first, before this.

How would you implement this in new regime [~busbey]? It changes FSHLog.  You'd do a derivative
or decorated FSHLog, SwitchingFSHLog?  Can get rid of stuff like the 'enabled' flag and checks.

Looking at last patch, pity we couldn't switch to the other writer when rolling log on current
writer (given rolling takes a while).  Looks like this was a consideration: "2241	   * NOTE:
Don't switch if there is a ongoing log roll. Most likely, this could be a redundant
2242	   * step."

Patch is worth a study. Pity has to be a syncmonitor but not sure how else you'd do it.

On keeping around edits, one implementation, rather than append the WAL directly as we do
 now, instead, kept the edits in a single list and then did bulk appends (IIRC, no advantage
doing bulk append over single appends). Edits stayed in List until syncs came back to say
it was ok let them go.  IIRC, it was not that much slower.  It was a little more involved
(was easier just doing the WAL append  immediately since then we were done) but it might be
worth considering having a single list of all outstanding edits on other side of the ring
buffer as store for edits in flight (downside would be extra thread coordination)

> Provide better write predictability
> -----------------------------------
>                 Key: HBASE-10278
>                 URL: https://issues.apache.org/jira/browse/HBASE-10278
>             Project: HBase
>          Issue Type: New Feature
>          Components: wal
>            Reporter: Himanshu Vashishtha
>            Assignee: Himanshu Vashishtha
>         Attachments: 10278-trunk-v2.1.patch, 10278-trunk-v2.1.patch, 10278-wip-1.1.patch,
Multiwaldesigndoc.pdf, SwitchWriterFlow.pptx
> Currently, HBase has one WAL per region server. 
> Whenever there is any latency in the write pipeline (due to whatever reasons such as
n/w blip, a node in the pipeline having a bad disk, etc), the overall write latency suffers.

> Jonathan Hsieh and I analyzed various approaches to tackle this issue. We also looked
at HBASE-5699, which talks about adding concurrent multi WALs. Along with performance numbers,
we also focussed on design simplicity, minimum impact on MTTR & Replication, and compatibility
with 0.96 and 0.98. Considering all these parameters, we propose a new HLog implementation
with WAL Switching functionality.
> Please find attached the design doc for the same. It introduces the WAL Switching feature,
and experiments/results of a prototype implementation, showing the benefits of this feature.
> The second goal of this work is to serve as a building block for concurrent multiple
WALs feature.
> Please review the doc.

This message was sent by Atlassian JIRA

View raw message