From dev-return-25581-apmail-hbase-dev-archive=hbase.apache.org@hbase.apache.org Mon Nov 28 20:36:17 2011 Return-Path: X-Original-To: apmail-hbase-dev-archive@www.apache.org Delivered-To: apmail-hbase-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DF7F3927F for ; Mon, 28 Nov 2011 20:36:16 +0000 (UTC) Received: (qmail 28104 invoked by uid 500); 28 Nov 2011 20:36:16 -0000 Delivered-To: apmail-hbase-dev-archive@hbase.apache.org Received: (qmail 28064 invoked by uid 500); 28 Nov 2011 20:36:16 -0000 Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hbase.apache.org Delivered-To: mailing list dev@hbase.apache.org Received: (qmail 28053 invoked by uid 99); 28 Nov 2011 20:36:16 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Nov 2011 20:36:16 +0000 X-ASF-Spam-Status: No, hits=3.4 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,URIBL_JP_SURBL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of sujee@sujee.net designates 209.85.161.169 as permitted sender) Received: from [209.85.161.169] (HELO mail-gx0-f169.google.com) (209.85.161.169) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Nov 2011 20:36:11 +0000 Received: by ggnq4 with SMTP id q4so1688169ggn.14 for ; Mon, 28 Nov 2011 12:35:50 -0800 (PST) Received: by 10.236.201.196 with SMTP id b44mr22367485yho.48.1322512550131; Mon, 28 Nov 2011 12:35:50 -0800 (PST) MIME-Version: 1.0 Received: by 10.101.109.3 with HTTP; Mon, 28 Nov 2011 12:35:29 -0800 (PST) From: Sujee Maniyam Date: Mon, 28 Nov 2011 12:35:29 -0800 Message-ID: Subject: need feedback on PerformanceEvaluation with presplit option test code To: hbase-user , dev@hbase.apache.org Content-Type: multipart/alternative; boundary=20cf305e25adc2f6dc04b2d1721e --20cf305e25adc2f6dc04b2d1721e Content-Type: text/plain; charset=ISO-8859-1 Hi All I have added a presplit option to PerformanceEvaluation class. I see the TestTable is created with splits. But when I run 'randomWrite' test (in MR mode) majority of the 'requests' are going to only one region server. Other region servers are busy as well, but catering to small number of requests. See this screen shot: http://www.sujee.net/tech/hbase-region-server-split2.png only devperf-sn2 is getting most of the requests (~20k). Other RS are only getting a few hundred requests. This is on : Hbase 0.90.4 cdh3u2 I'd like some feedback on why this is happening. The code is here: http://pastebin.com/N18jWV4Q notable changes : - line 471 : checktable - line 509 : getSplits jar : http://www.sujee.net/tech/sujee.jar how to reproduce: 1) Have sujee.jar in HBASE_CLASSPATH. One approach is to put 'sujee.jar' in hbase/lib dir and restarting Hbase. 2) run hbase org.apache.hadoop.hbase.PerformanceEvaluation2 --presplit=10 randomWrite 10 - I have made a a copy of 'Performance Evaluation' class for easy testing - presplit option takes a number (splits) 3) watch the requests on HBase Master UI thanks very much Sujee http://sujee.net --20cf305e25adc2f6dc04b2d1721e--