hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Trivial Update of "DataProcessingBenchmarks" by udanax
Date Tue, 08 Jan 2008 08:41:38 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by udanax:
http://wiki.apache.org/lucene-hadoop/DataProcessingBenchmarks

------------------------------------------------------------------------------
  
  SQL > select ipaddress, count(*) from access_log group by ipaddress order by count(*)
desc limit 0,100;
  [[BR]]''σ ,,count. ipaddress,, (τ ,,count,, (γ ,,count(ipaddress). ipaddress,, (access_log)))''
+ 
+  * After HADOOP-2480 done, Hbase will be join to benchmarks.
  
  ||<bgcolor="#E5E5E5">||<bgcolor="#E5E5E5">!MySql 5.0.27, MyISAM||<bgcolor="#E5E5E5">Hadoop-0.15.0
Release||
  ||<bgcolor="#E5E5E5">Index ||index ||non-index||
@@ -14, +16 @@

  ||<bgcolor="#E5E5E5">Results ||100 ||100||
  ||<bgcolor="#E5E5E5">Time  ||3.715 sec ||112.03 sec||
  
-  * After HADOOP-2480 done, Hbase will be join to benchmarks.
+ ==== Processing Flow ====
+ 
+  * Map was used for extract the IP address of the client requesting the web page.
+  * Reduce was used for summation.
  
  {{{
  bash# ./bin/hadoop jar ./log_examples.jar loganalysis -m 1000 -r 1000 udanax/logfiles udanax/rank
100
@@ -44, +49 @@

  08/01/08 16:15:35 INFO mapred.JobClient:     Reduce output records=201330
  }}}
  
+  * Map/Reduce was used for sort by count.
-  * Map was used for extract the IP address of the client requesting the web page.
-  * Reduce was used for summation.
  
  {{{
  * Sort by Connection Time Count Map/Reduce Job start.
@@ -70, +74 @@

  08/01/08 16:33:55 INFO mapred.JobClient:     Reduce input groups=19270
  08/01/08 16:33:55 INFO mapred.JobClient:     Reduce input records=84069
  08/01/08 16:33:55 INFO mapred.JobClient:     Reduce output records=200
- }}}
  
-  * Map/Reduce was used for sort by count.
- 
- {{{
  ------------------------------------
  * Top 100 connector list :
  +--------------+-------------------+

Mime
View raw message