hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Trivial Update of "Hbase/NewFileFormat/Performance" by stack
Date Fri, 06 Feb 2009 23:01:14 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by stack:
http://wiki.apache.org/hadoop/Hbase/NewFileFormat/Performance

The comment on the change is:
Adding in rfile w/ 64k buffers

------------------------------------------------------------------------------
  Numbers comparing MapFile and RFile (TFile+mods has dropped from the running for the moment
anyways).  The code used running tests is available over in [http://github.com/ryanobjc/hbase-rfile/tree/rfile
github].  I did following on local filesystem and on 4node hdfs:
  
- {{{$ ./bin/hadoop org.apache.hadoop.hbase.MapFilePerformanceEvaluation ;  ./bin/hadoop org.apache.hadoop.hbase.RFilePerformanceEvaluation}}}
+ {{{$ ./bin/hadoop org.apache.hadoop.hbase.MapFilePerformanceEvaluation
+ $ ./bin/hadoop org.apache.hadoop.hbase.RFilePerformanceEvaluation}}}
+ 
+ For more context, see [wiki:Hbase/NewFileFormat New File Format].
  
  == Local Filesystem ==
  Macosx, 10 byte cells and keys.
@@ -151, +154 @@

  09/02/06 22:10:48 INFO hbase.MapFilePerformanceEvaluation: Running SequentialReadBenchmark
for 100000 rows.
  09/02/06 22:10:50 INFO hbase.MapFilePerformanceEvaluation: Running SequentialReadBenchmark
for 100000 rows took 2625ms.}}}
  
+ == HDFS 1k cells ==
+ 
+ === MapFile ===
+ {{{09/02/06 22:28:58 INFO hbase.MapFilePerformanceEvaluation: Running SequentialWriteBenchmark
for 100000 rows.
+ 09/02/06 22:28:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
+ 09/02/06 22:28:58 INFO compress.CodecPool: Got brand-new compressor
+ 09/02/06 22:28:58 INFO compress.CodecPool: Got brand-new compressor
+ 09/02/06 22:29:13 INFO hbase.MapFilePerformanceEvaluation: Running SequentialWriteBenchmark
for 100000 rows took 14915ms.
+ 09/02/06 22:29:13 INFO hbase.MapFilePerformanceEvaluation: Running UniformRandomSmallScan
for 100000 rows.
+ 09/02/06 22:29:46 INFO hbase.MapFilePerformanceEvaluation: Running UniformRandomSmallScan
for 100000 rows took 32558ms.
+ 09/02/06 22:29:46 INFO hbase.MapFilePerformanceEvaluation: Running UniformRandomReadBenchmark
for 100000 rows.
+ 09/02/06 22:33:55 INFO hbase.MapFilePerformanceEvaluation: Running UniformRandomReadBenchmark
for 100000 rows took 249211ms.
+ 09/02/06 22:33:55 INFO hbase.MapFilePerformanceEvaluation: Running GaussianRandomReadBenchmark
for 100000 rows.
+ 09/02/06 22:37:49 INFO hbase.MapFilePerformanceEvaluation: Running GaussianRandomReadBenchmark
for 100000 rows took 234521ms.
+ 09/02/06 22:37:49 INFO hbase.MapFilePerformanceEvaluation: Running SequentialReadBenchmark
for 100000 rows.
+ 09/02/06 22:37:52 INFO hbase.MapFilePerformanceEvaluation: Running SequentialReadBenchmark
for 100000 rows took 2827ms.}}}
+ 
+ === RFile 64k buffers ===
+ {{{09/02/06 22:37:53 INFO hbase.RFilePerformanceEvaluation: Running SequentialWriteBenchmark
for 100000 rows.
+ 09/02/06 22:37:56 INFO hbase.RFilePerformanceEvaluation: Running SequentialWriteBenchmark
for 100000 rows took 3083ms.
+ 09/02/06 22:37:56 INFO hbase.RFilePerformanceEvaluation: Running UniformRandomSmallScan
for 100000 rows.
+ 09/02/06 22:38:24 INFO hbase.RFilePerformanceEvaluation: Running UniformRandomSmallScan
for 100000 rows took 27405ms.
+ 09/02/06 22:38:24 INFO hbase.RFilePerformanceEvaluation: Running UniformRandomReadBenchmark
for 100000 rows.
+ 09/02/06 22:41:24 INFO hbase.RFilePerformanceEvaluation: Running UniformRandomReadBenchmark
for 100000 rows took 180332ms.
+ 09/02/06 22:41:24 INFO hbase.RFilePerformanceEvaluation: Running GaussianRandomReadBenchmark
for 100000 rows.
+ 09/02/06 22:44:20 INFO hbase.RFilePerformanceEvaluation: Running GaussianRandomReadBenchmark
for 100000 rows took 175614ms.
+ 09/02/06 22:44:20 INFO hbase.RFilePerformanceEvaluation: Running SequentialReadBenchmark
for 100000 rows.
+ 09/02/06 22:44:23 INFO hbase.RFilePerformanceEvaluation: Running SequentialReadBenchmark
for 100000 rows took 2840ms.}}}
+ 

Mime
View raw message