hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-10278) Provide better write predictability
Date Thu, 24 Apr 2014 08:19:19 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13979439#comment-13979439
] 

Jonathan Hsieh edited comment on HBASE-10278 at 4/24/14 8:17 AM:
-----------------------------------------------------------------

Currently trunk has some correctness problems when running the ITMTTR (having to do with killing
the rs hosting meta).

However running trunk and a modified version with this patch applied and on by default we
see significantly worse worst case recovery time when the target RS is killed. 

{code}
		kill master			kill rs				move regions			
		AVE,STD 99	AVE,STD 99.99	AVE,STD 99	AVE,STD 99.99	AVE,STD 99	AVE,STD 99.99
mhlog	admin	18302.5	122.8	18302.5	122.8	2.1	0.3	51.9	10.4	2.0	0.0	34.2	7.6
	put	5.1	0.3	117.9	95.3	5.1	0.3	37647.2	9888.0	5.6	0.5	169.4	30.3
	scan	2.0	0.0	24.1	15.7	3.7	0.9	36131.2	13245.5	2.0	0.0	45.2	14.7
trunk	admin	18557.3	357.2	18557.3	357.2	2.1	0.3	41.8	7.2	2.0	0.0	31.7	4.7
	put	5.4	0.6	79.4	92.0	5.1	0.3	735.2	673.7	5.0	0.0	130.4	13.9
	scan	2.0	0.0	27.0	15.8	2.4	0.7	165.7	138.3	2.0	0.5	39.9	9.4
{code}


was (Author: jmhsieh):
Currently trunk has some correctness problems when running the ITMTTR (having to do with killing
the rs hosting meta).

However running trunk and a modified version with this patch applied and on by default we
see significantly worse recovery time when the target RS is killed. 

{code}
		kill master			kill rs				move regions			
		AVE,STD 99	AVE,STD 99.99	AVE,STD 99	AVE,STD 99.99	AVE,STD 99	AVE,STD 99.99
mhlog	admin	18302.5	122.8	18302.5	122.8	2.1	0.3	51.9	10.4	2.0	0.0	34.2	7.6
	put	5.1	0.3	117.9	95.3	5.1	0.3	37647.2	9888.0	5.6	0.5	169.4	30.3
	scan	2.0	0.0	24.1	15.7	3.7	0.9	36131.2	13245.5	2.0	0.0	45.2	14.7
trunk	admin	18557.3	357.2	18557.3	357.2	2.1	0.3	41.8	7.2	2.0	0.0	31.7	4.7
	put	5.4	0.6	79.4	92.0	5.1	0.3	735.2	673.7	5.0	0.0	130.4	13.9
	scan	2.0	0.0	27.0	15.8	2.4	0.7	165.7	138.3	2.0	0.5	39.9	9.4
{code}

> Provide better write predictability
> -----------------------------------
>
>                 Key: HBASE-10278
>                 URL: https://issues.apache.org/jira/browse/HBASE-10278
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Himanshu Vashishtha
>            Assignee: Himanshu Vashishtha
>         Attachments: 10278-trunk-v2.1.patch, 10278-trunk-v2.1.patch, 10278-wip-1.1.patch,
Multiwaldesigndoc.pdf, SwitchWriterFlow.pptx
>
>
> Currently, HBase has one WAL per region server. 
> Whenever there is any latency in the write pipeline (due to whatever reasons such as
n/w blip, a node in the pipeline having a bad disk, etc), the overall write latency suffers.

> Jonathan Hsieh and I analyzed various approaches to tackle this issue. We also looked
at HBASE-5699, which talks about adding concurrent multi WALs. Along with performance numbers,
we also focussed on design simplicity, minimum impact on MTTR & Replication, and compatibility
with 0.96 and 0.98. Considering all these parameters, we propose a new HLog implementation
with WAL Switching functionality.
> Please find attached the design doc for the same. It introduces the WAL Switching feature,
and experiments/results of a prototype implementation, showing the benefits of this feature.
> The second goal of this work is to serve as a building block for concurrent multiple
WALs feature.
> Please review the doc.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message