hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-16890) Analyze the performance of AsyncWAL and fix the same
Date Fri, 04 Nov 2016 02:04:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634943#comment-15634943
] 

Duo Zhang edited comment on HBASE-16890 at 11/4/16 2:04 AM:
------------------------------------------------------------

So the key point is the HDFS cluster must be single node? I run the WALPE on a standard HDFS
cluster with replication set to 1. I can see this log in the output

{noformat}
2016-11-03 20:58:16,126 INFO  [WALPerfEval.logRoller] wal.AbstractFSWAL: Slow sync cost: 104
ms, current pipeline: [DatanodeInfoWithStorage[10.108.18.25:35402,DS-eb75553e-c64d-42a3-ab43-a772a74c7b51,DISK]]
{noformat}

So I think it worked? And I set block size to 1GB and roll size to 512M to prevent FSHLog
roll a new block, the result is still same, FSHLog is the slowest...

And my HDFS version is 2.7.3.


was (Author: apache9):
So the key point is the HDFS cluster must be single node? I run the WALPE on a standard HDFS
cluster with replication set to 1. I can see this log in the output

{noformat}
2016-11-03 20:58:16,126 INFO  [WALPerfEval.logRoller] wal.AbstractFSWAL: Slow sync cost: 104
ms, current pipeline: [DatanodeInfoWithStorage[10.108.18.25:35402,DS-eb75553e-c64d-42a3-ab43-a772a74c7b51,DISK]]
{noformat}

So I think it worked? And I set block size to 1GB and roll size to 512M to prevent FSHLog
roll a new block, the result is still same, FSHLog is the slowest...

> Analyze the performance of AsyncWAL and fix the same
> ----------------------------------------------------
>
>                 Key: HBASE-16890
>                 URL: https://issues.apache.org/jira/browse/HBASE-16890
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>
>         Attachments: AsyncWAL_disruptor.patch, AsyncWAL_disruptor_1 (2).patch, AsyncWAL_disruptor_3.patch,
AsyncWAL_disruptor_3.patch, AsyncWAL_disruptor_4.patch, HBASE-16890-rc-v2.patch, HBASE-16890-remove-contention-v1.patch,
HBASE-16890-remove-contention.patch, Screen Shot 2016-10-25 at 7.34.47 PM.png, Screen Shot
2016-10-25 at 7.39.07 PM.png, Screen Shot 2016-10-25 at 7.39.48 PM.png, async.svg, classic.svg,
contention.png, contention_defaultWAL.png
>
>
> Tests reveal that AsyncWAL under load in single node cluster performs slower than the
Default WAL. This task is to analyze and see if we could fix it.
> See some discussions in the tail of JIRA HBASE-15536.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message