hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sujee Maniyam <su...@sujee.net>
Subject need feedback on PerformanceEvaluation with presplit option test code
Date Mon, 28 Nov 2011 20:35:29 GMT
Hi All

I have added a presplit option to PerformanceEvaluation class.

I see the TestTable is created with splits.  But when I run 'randomWrite'
test (in MR mode)  majority of the 'requests' are going to only one region
server.  Other region servers are busy as well, but catering to small
number of requests.

See this screen shot:
http://www.sujee.net/tech/hbase-region-server-split2.png
only devperf-sn2  is getting most of the requests (~20k).  Other RS are
only getting a few hundred requests.

This is on : Hbase 0.90.4 cdh3u2

I'd like some feedback on why this is happening.

The code is here:  http://pastebin.com/N18jWV4Q
notable changes :
   - line 471 : checktable
   - line 509 : getSplits

jar : http://www.sujee.net/tech/sujee.jar

how to reproduce:

1) Have sujee.jar in HBASE_CLASSPATH.
One approach is to put 'sujee.jar' in hbase/lib dir and restarting Hbase.

2) run
hbase  org.apache.hadoop.hbase.PerformanceEvaluation2  --presplit=10
randomWrite 10

- I have made a a copy of 'Performance Evaluation' class for easy testing
-  presplit option takes a number (splits)

3) watch the requests on HBase Master UI

thanks very much
Sujee
http://sujee.net

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message