hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16890) Analyze the performance of AsyncWAL and fix the same
Date Sat, 05 Nov 2016 00:33:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15638184#comment-15638184
] 

stack commented on HBASE-16890:
-------------------------------

I ran the tests a few times and results consistent. Looking in FSHLog run w/ JFR, I see more
points of contention reported -- inside DFSClient. It uses maybe 25% more CPU probably because
of the upped throughput.  Otherwise, looking w/ JFR nothing jumps out. Let me put up pictures
of the 'hot methods' It is almost as though FSHLog is doing more work (The top consumers are
the WALPE random generation... we should fix that).

The FSHLog must have a better 'flow' going on. Here is histograms for FSHLog:

{code}

-- Histograms ------------------------------------------------------------------
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.latencyHistogram.nanos
             count = 8461245
               min = 838241
               max = 115799121
              mean = 2696785.63
            stddev = 6486391.73
            median = 2199081.00
              75% <= 2571547.00
              95% <= 3237948.00
              98% <= 3621166.00
              99% <= 5216818.00
            99.9% <= 115799121.00
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncCountHistogram.countPerSync
             count = 412764
               min = 1
               max = 86
              mean = 21.04
            stddev = 16.98
            median = 17.00
              75% <= 34.00
              95% <= 53.00
              98% <= 58.00
              99% <= 62.00
            99.9% <= 86.00
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncHistogram.nanos-between-syncs
             count = 412764
               min = 405379
               max = 129879546
              mean = 1680258.91
            stddev = 7343616.88
            median = 1127074.00
              75% <= 1448611.00
              95% <= 1812916.00
              98% <= 1978098.00
              99% <= 2150048.00
            99.9% <= 122766311.00

-- Meters ----------------------------------------------------------------------
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.appendMeter.bytes
             count = 59144801550
         mean rate = 244727411.22 events/second
     1-minute rate = 245882558.80 events/second
     5-minute rate = 199668915.99 events/second
    15-minute rate = 166822622.37 events/second
org.apache.hadoop.hbase.wal.WALPerformanceEvaluation.syncMeter.syncs
             count = 412764
         mean rate = 1707.90 events/second
     1-minute rate = 1715.17 events/second
     5-minute rate = 1342.77 events/second
    15-minute rate = 1077.71 events/second
{code}

Let me get them for asyncwal...

> Analyze the performance of AsyncWAL and fix the same
> ----------------------------------------------------
>
>                 Key: HBASE-16890
>                 URL: https://issues.apache.org/jira/browse/HBASE-16890
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>
>         Attachments: AsyncWAL_disruptor.patch, AsyncWAL_disruptor_1 (2).patch, AsyncWAL_disruptor_3.patch,
AsyncWAL_disruptor_3.patch, AsyncWAL_disruptor_4.patch, AsyncWAL_disruptor_6.patch, HBASE-16890-rc-v2.patch,
HBASE-16890-rc-v3.patch, HBASE-16890-remove-contention-v1.patch, HBASE-16890-remove-contention.patch,
Screen Shot 2016-10-25 at 7.34.47 PM.png, Screen Shot 2016-10-25 at 7.39.07 PM.png, Screen
Shot 2016-10-25 at 7.39.48 PM.png, async.svg, classic.svg, contention.png, contention_defaultWAL.png
>
>
> Tests reveal that AsyncWAL under load in single node cluster performs slower than the
Default WAL. This task is to analyze and see if we could fix it.
> See some discussions in the tail of JIRA HBASE-15536.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message