hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16890) Analyze the performance of AsyncWAL and fix the same
Date Thu, 03 Nov 2016 11:10:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15632400#comment-15632400

ramkrishna.s.vasudevan commented on HBASE-16890:

bq.AsyncFSWAL with ram's patch test hang, can not finish
Which patch did you take!!!
I can try with 1 thread not sure why it hanged. In my tests my patch performed better with
100 threads. Ya there is no network latency at all in our set up. Everything runs in the same
machine. So we get much  better thro put (ops/sec with WALPE). In your set up DN and NN are
in different machine?
But there is more to it. Hang on.
I tried testing with One node real cluster with PE tool. Loaded 40G of data with 50 threads
and each row having 50 cols.
Min: 524525ms   Max: 546482ms   Avg: 540959ms

AsycnWAL with Ram's patch
Min: 992706ms   Max: 1010504ms  Avg: 1006560ms

With Duo's patch
Min: 992568ms   Max: 1020693ms  Avg: 1016597ms
So as I said earlier and as Duo mentions is it not worth to test in single node machines?

> Analyze the performance of AsyncWAL and fix the same
> ----------------------------------------------------
>                 Key: HBASE-16890
>                 URL: https://issues.apache.org/jira/browse/HBASE-16890
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>         Attachments: AsyncWAL_disruptor.patch, AsyncWAL_disruptor_1 (2).patch, AsyncWAL_disruptor_3.patch,
AsyncWAL_disruptor_3.patch, AsyncWAL_disruptor_4.patch, HBASE-16890-remove-contention-v1.patch,
HBASE-16890-remove-contention.patch, Screen Shot 2016-10-25 at 7.34.47 PM.png, Screen Shot
2016-10-25 at 7.39.07 PM.png, Screen Shot 2016-10-25 at 7.39.48 PM.png, async.svg, classic.svg,
contention.png, contention_defaultWAL.png
> Tests reveal that AsyncWAL under load in single node cluster performs slower than the
Default WAL. This task is to analyze and see if we could fix it.
> See some discussions in the tail of JIRA HBASE-15536.

This message was sent by Atlassian JIRA

View raw message