hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Update of "Hbase/PerformanceEvaluation" by stack
Date Mon, 11 Jun 2007 05:09:05 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by stack:
http://wiki.apache.org/lucene-hadoop/Hbase/PerformanceEvaluation

The comment on the change is:
Add initial test figures for single region server

------------------------------------------------------------------------------
  = Testing HBase Performance and Scalability =
+ 
+ == Tool Description ==
  
  [https://issues.apache.org/jira/browse/HADOOP-1476 HADOOP-1476] adds to HBase {{{src/test}}}
the script {{{org.apache.hadoop.hbase.PerformanceEvaluation}}}.  It runs the tests described
in ''Performance Evaluation'', Section 7 of the [http://labs.google.com/papers/bigtable.html
BigTable paper].  See the citation for test descriptions.  They will not be described below.
The script is useful evaluating HBase performance and how well it scales as we add region
servers.
  
@@ -61, +63 @@

  $ ${HBASE_HOME}/bin/hbase ciient deleteTable TestTable
  }}}
  
- Some first figures in advance of any profiling of the current state of the HBase code (on
Fri Jun 8 2007) would seem to indicate that HBase runs at about an order-of-magnitude slower
than whats reported in the BigTable paper running on similiar hardware (more on this to follow).
  
+ == One Region Server on June 8th, 2007 ==
+ Here are some first figures for HBase in advance of any profiling taken June 8, 2007
+ 
+ This first test ran on a mini test cluster of four machines only: not the 1768 of the !BigTable
paper.  Each node had 8G of RAM and 2x dual-core 2Ghz Opterons.  Every member ran a HDFS datanode.
 One node ran the namenode and the HBase master, another the region server and a third an
instance of the !PerformanceEvaluation script configured to run one client.  Clients write
~1GB of data: One million rows, each row has a single column whose value is 1000 randomly-generated
bytes (See the BigTable paper for a better description).
+ 
+ 
+ ||Experiment||HBase||!BigTable||
+ ||random reads ||68||1212||
+ ||random reads (mem)||Not implemented||10811||
+ ||random writes||847||8850||
+ ||sequential reads||301||4425||
+ ||sequential writes||850||8547||
+ ||scans||3063||15385||
+ 
+ The above table lists how many 1000-byte rows read/written per second.  The !BigTable values
are from '1' Tablet Server column of Figure 6 of the !BigTable paper.
+ 
+ Except for scanning, we seem to be an order of magnitude off at the moment.  Watching the
region server during the write tests, it was lightly loaded.  At a minimum, there would appear
to be issues with liveness/synchronization in need of fixing.
+ 
+ More to follow after more analysis.
+ 

Mime
View raw message