Return-Path: X-Original-To: apmail-hbase-dev-archive@www.apache.org Delivered-To: apmail-hbase-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 11157108C7 for ; Sat, 15 Nov 2014 03:43:58 +0000 (UTC) Received: (qmail 35764 invoked by uid 500); 15 Nov 2014 03:43:57 -0000 Delivered-To: apmail-hbase-dev-archive@hbase.apache.org Received: (qmail 35673 invoked by uid 500); 15 Nov 2014 03:43:57 -0000 Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hbase.apache.org Delivered-To: mailing list dev@hbase.apache.org Received: (qmail 35662 invoked by uid 99); 15 Nov 2014 03:43:57 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 15 Nov 2014 03:43:57 +0000 Received: from mail-la0-f51.google.com (mail-la0-f51.google.com [209.85.215.51]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 06F181A02FC for ; Sat, 15 Nov 2014 03:42:55 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id q1so16102383lam.38 for ; Fri, 14 Nov 2014 19:43:51 -0800 (PST) X-Received: by 10.112.180.198 with SMTP id dq6mr11532491lbc.56.1416023031614; Fri, 14 Nov 2014 19:43:51 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.209.145 with HTTP; Fri, 14 Nov 2014 19:43:11 -0800 (PST) From: Andrew Purtell Date: Fri, 14 Nov 2014 19:43:11 -0800 Message-ID: Subject: Comparing the performance of 0.98.8 RC1 and 0.98.0 using YCSB To: "dev@hbase.apache.org" Content-Type: multipart/alternative; boundary=001a11c26b42cb13d40507dd8d41 --001a11c26b42cb13d40507dd8d41 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Comparing the performance of 0.98.8 RC1 and 0.98.0 using YCSB from https://github.com/apurtell/ycsb/tree/new_hbase_client. In summary, there are minor differences, generally improvements, and this test didn't detect performance regression when looking at operation latencies. Note 0.98.8RC2 only has HBASE-11698 and HBASE-12478 in addition to earlier changes, neither of which will alter these results. Hardware and Versions Hadoop 2.4.1 HBase 0.98.8-hadoop2 RC1 11x EC2 c3.8xlarge: 1 master, 5 slaves, 5 test clients 32 cores 60 GB RAM 2 x 320 GB directly attached SSD NameNode: 4 GB heap DataNode: 1 GB heap Master: 1 GB heap RegionServer: 8 GB heap, 24 GB bucket cache offheap engine Methodology Setup: 0. Start cluster 1. shell: create "seed", { NAME=3D>"u", COMPRESSION=3D>"snappy" } 2. YCSB: Preload 100 million rows into table "seed" 3. shell: flush "seed" ; compact "seed" 4. Wait for compaction to complete 5. shell: create_snapshot "seed", "seed_snap" 6. shell: disable "seed" For each test: 7. shell: clone_snapshot "seed_snap", "test" 8. YCSB: Run 5x clients each with -p operationcount=3D4000000 -th= reads 100 -target 20000 (aggregate 100k/ops/sec) 9. shell: disable "test" 10. shell: drop "test" =E2=80=8BI then ran one more workload for each version with the lightweight-java-profiler agent installed on one regionserver and generated flame graphs from the resulting traces, including the neat new differential type. When looking at the differential flame graphs, you'll note the deltas are positive. This is the difference between time sampled in the relevant call stack on 0.98.0 versus 0.98.8, a positive number means _less_ time was spent by 0.98.8 there than 0.98.0. It's also likely that testing with higher loads (this test was only with 100k/ops/sec over a 5 slave cluster) will present more significant differences. For more information, see http://www.brendangregg.com/blog/2014-11-09/differential-flame-graphs.html =E2=80=8BResults =E2=80=8BYCSB summary: http://people.apache.org/~apurtell/results-ycsb-0.98.8/ycsb-0.98.0-vs-0.98.= 8.pdf Flame Graphs 0.98.0 A: http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_0.98.0.svg 0.98.8 A: http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_0.98.8.svg Difference: http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_diff.svg 0.98.0 B: http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_0.98.0.svg 0.98.8 B: http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_0.98.8.svg Difference: http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_diff.svg 0.98.0 C: http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_0.98.0.svg 0.98.8 C: http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_0.98.8.svg Difference: http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_diff.svg 0.98.0 D: http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_0.98.0.svg 0.98.8 D: http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_0.98.8.svg Difference: http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_diff.svg 0.98.0 E: http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_0.98.0.svg 0.98.8 E: http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_0.98.8.svg Difference: http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_diff.svg 0.98.0 F: http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_0.98.0.svg 0.98.8 F: http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_0.98.8.svg Difference: http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_diff.svg --=20 Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) --001a11c26b42cb13d40507dd8d41--