Mailing-List: contact pig-commits-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: pig-dev@hadoop.apache.org
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: Apache Wiki <wikidiffs@apache.org>
To: pig-commits@incubator.apache.org
Date: Tue, 24 Feb 2009 19:18:34 -0000
Message-ID: <20090224191834.16558.80282@aurora.apache.org>
Subject: [Pig Wiki] Update of "PigMix" by AlanGates

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.

The following page has been changed by AlanGates:
http://wiki.apache.org/pig/PigMix

------------------------------------------------------------------------------
  
  == Runs ==
  
- == November 19, 2008 ==
- PigMix was run on a 25 machine hadoop cluster.  The cluster was running hadoop version 0.18.1.  The tests were run against two
+ The following table includes runs done of the pig mix.  All of these runs have been done on a cluster with 26 slaves plus one machine acting as the name node and job tracker.  The cluster was running 
+ hadoop version 0.18.1.  (TODO:  Need to get specific hardware info on those machines).  
+ 
+ The tests were run against two
  versions of pig:  top of trunk, and top of types branch both as of Nov 21 2008.
  
- So far only the latency tests have been run, not the scalability tests.  Equivalent map reduce programs have not yet been run.
- 
  The tests were run three times for each version and the results averaged.
  
- || Query || Top of Trunk || Top of Types Branch ||
- || L1 explode || 261 || 283 ||
- || L2 fr join || 1665 || 253 ||
- || L3 join || 1912 || 320 ||
- || L4 distinct agg || 254 || 193 ||
- || L5 anti-join || 1535 || 281 ||
- || L6 large group by key || 294 || 226 ||
- || L7 nested split || 243 || 204 ||
- || L8 group all || 462 ||  194 ||
- || L9 order by 1 field || 5294 || 867 ||
- || L10 order by multiple fields || 1403 || 565 ||
- || L11 distinct + union || 316 || 255 ||
- || L12 multi-store || fails || 781 ||
+ tot = top of trunk
+ totb = top of types branch
+ 
+ || Version                      || Map Reduce Java Code || tot 11/21/08 || totb 11/21/08 || totb 1/20/09 ||
+ || Date Run                     || 11/22/08             || 11/21/08     || 11/21/08      || 1/20/09      ||
+ || L1 explode                   || 116                  || 261          || 283           || 218          ||
+ || L2 fr join                   ||  41                  || 1665         || 253           || 168          ||
+ || L3 join                      ||  97                  || 1912         || 320           || 258          ||
+ || L4 distinct agg              ||  68                  || 254          || 193           || 110          ||
+ || L5 anti-join                 ||  90                  || 1535         || 281           || 209          ||
+ || L6 large group by key        ||  61                  || 294          || 226           || 126          ||
+ || L7 nested split              ||  72                  || 243          || 204           || 107          ||
+ || L8 group all                 ||  56                  || 462          || 194           || 104          ||
+ || L9 order by 1 field          || 286                  || 5294         || 867           || 851          ||
+ || L10 order by multiple fields || 634                  || 1403         || 565           || 469          ||
+ || L11 distinct + union         || 120                  || 316          || 255           || 164          ||
+ || L12 multi-store              || 150                  || fails        || 781           || 499          ||
+ || Total time                   || 1791                 || 13638        || 4420          || 3284         ||
+ || Compared to hadoop           || 1.0                  || 7.6          || 2.5           || 1.8          ||
+ 
+ The totb run of 1/20/09 includes the change to make BufferedPositionedInputStream use a buffer instead of relying on hadoop to buffer.
  
  == Features Tested ==
  Based on a sample of user queries, PigMix includes tests for the following features.