hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-5699) Run with > 1 WAL in HRegionServer
Date Wed, 10 Dec 2014 19:19:16 GMT

     [ https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sean Busbey updated HBASE-5699:
    Attachment: HBASE-5699_write_iops_multiwal-6_10,50,120,190,260,330,400_threads.tiff

Attaching results from some initial testing using WALPerformanceEvaluation with a sync-heavy

If anyone has other measurements they'd like to see, or a different workload expressed (perhaps
to look at limits for pushing bytes given larger edits with fewer syncs), please let me know.

h5. Overview
command used was
bin/hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation -threads ${threads} -regions
$(((threads+1)/2)) -roll 100000 -iterations 1000000 -verify 

with # threads varied and # regions at ceil(threads/2). default is sync-per-write.

Test rig is a physical cluster with 4 data nodes and a total of 20 data disks (5 per node).
Test client was run on a separate non-loaded host. HDFS 2.5.0-cdh5.2.0

* {{.bz2}} files are the complete logs from the described group of runs.
* "upstream" and "wals-1" data is from prior to HBASE-12655, so run metrics other than the
final benchmark results aren't comparable to later.
* There's an image showing total write iops across the cluster for each of the sets of test
* hbase-5699_total_throughput_sync_heavy.txt has the final benchmark log from each of the
runs, so you can quickly look across successive runs.
* hbase-5699_multiwal_400-threads_stats_sync_heavy.txt has the run metrics from just the final
test of the multiwal options

h5. upstream vs multiwal-1

If you look at the two charts "HBASE-5699_write_iops_upstream_1_to_200_threads.tiff" and "HBASE-5699_write_iops_multiwal-1_1_to_200_threads.tiff",
they behave roughly the same modulo noise. (the only difference in the two happens during
region set up, which shouldn't be reflected here.)

They also level off on ability to push more through the pipeline at near the limit for iops
given the 3 disks in the single pipeline.

h5. increasing number of pipelines
If you look at each of the HBASE-5699_write_iops_multiwal-X_10,50,120,190,260,330,400_threads.tiff
charts, as we ramp up the number of writers we manage to push more overall activity through
the cluster.

It's not a linear gain because splitting out the pipelines means that we do more overall syncs
since fewer of them get obviated by our sync grouping.

In this test, expanding from 2 to 4 or 6 pipelines didn't provide much benefit because at
up to 400 concurrent sync-heavy writers we just get to maxing out the number of iops that
can be done with 2 pipelines.

> Run with > 1 WAL in HRegionServer
> ---------------------------------
>                 Key: HBASE-5699
>                 URL: https://issues.apache.org/jira/browse/HBASE-5699
>             Project: HBase
>          Issue Type: Improvement
>          Components: Performance, wal
>            Reporter: binlijin
>            Assignee: Sean Busbey
>            Priority: Critical
>         Attachments: HBASE-5699.3.patch.txt, HBASE-5699_write_iops_multiwal-1_1_to_200_threads.tiff,
HBASE-5699_write_iops_multiwal-2_10,50,120,190,260,330,400_threads.tiff, HBASE-5699_write_iops_multiwal-4_10,50,120,190,260,330,400_threads.tiff,
HBASE-5699_write_iops_multiwal-6_10,50,120,190,260,330,400_threads.tiff, HBASE-5699_write_iops_upstream_1_to_200_threads.tiff,
PerfHbase.txt, hbase-5699_multiwal_400-threads_stats_sync_heavy.txt, hbase-5699_total_throughput_sync_heavy.txt,
results-hbase5699-upstream.txt.bz2, results-hbase5699-wals-1.txt.bz2, results-updated-hbase5699-wals-2.txt.bz2,
results-updated-hbase5699-wals-4.txt.bz2, results-updated-hbase5699-wals-6.txt.bz2

This message was sent by Atlassian JIRA

View raw message