incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Friedrich" <m...@mafr.de>
Subject Review Request: Add PGroupedTable.combineValues(Aggregator) and Aggregators.
Date Sun, 04 Nov 2012 12:33:33 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7858/
-----------------------------------------------------------

Review request for crunch.


Description
-------

Here's a patch to test ReviewBoard and to give you an impression of the new Aggregators class
and the changes this would require. I have simplified things even further than I originally
planned, so I need some feedback if I'm on the right track here.

Even when approved, I will not commit this immediately. There are a few things that need to
be done that I'll discuss on crunch-dev.


Diffs
-----

  crunch-contrib/src/main/java/org/apache/crunch/contrib/bloomfilter/BloomFilterFactory.java
825b445 
  crunch-examples/src/main/java/org/apache/crunch/examples/AverageBytesByIP.java 8abbb73 
  crunch-examples/src/main/java/org/apache/crunch/examples/TotalBytesByIP.java 44776ea 
  crunch-examples/src/main/java/org/apache/crunch/examples/WordAggregationHBase.java 691721d

  crunch/src/it/java/org/apache/crunch/CollectionsIT.java 0d5803e 
  crunch/src/it/java/org/apache/crunch/MapsIT.java 5b3187b 
  crunch/src/it/java/org/apache/crunch/WordCountIT.java 5124663 
  crunch/src/it/java/org/apache/crunch/fn/AggregatorsIT.java PRE-CREATION 
  crunch/src/it/java/org/apache/crunch/io/avro/AvroWritableIT.java cbb7fde 
  crunch/src/it/java/org/apache/crunch/lib/CogroupIT.java b6f5029 
  crunch/src/it/java/org/apache/crunch/test/Tests.java PRE-CREATION 
  crunch/src/it/resources/org/apache/crunch/fn/AggregatorsITData/ints.txt PRE-CREATION 
  crunch/src/main/java/org/apache/crunch/PGroupedTable.java e727b70 
  crunch/src/main/java/org/apache/crunch/fn/Aggregators.java PRE-CREATION 
  crunch/src/main/java/org/apache/crunch/impl/mem/collect/MemGroupedTable.java 0ee4c3f 
  crunch/src/main/java/org/apache/crunch/impl/mr/collect/PGroupedTableImpl.java fee381d 
  crunch/src/main/java/org/apache/crunch/lib/Aggregate.java f28cca4 
  crunch/src/test/java/org/apache/crunch/fn/AggregatorsTest.java PRE-CREATION 
  pom.xml 882ada1 

Diff: https://reviews.apache.org/r/7858/diff/


Testing
-------

Integration test works, as did a simple example on a single node cluster. Added an integration
test and unit tests based on CombineFnTest.


Thanks,

Matthias Friedrich


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message