incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills" <jwi...@cloudera.com>
Subject Re: Review Request: Add PGroupedTable.combineValues(Aggregator) and Aggregators.
Date Mon, 12 Nov 2012 20:34:54 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7858/#review13369
-----------------------------------------------------------

Ship it!


Ship It!

- Josh Wills


On Nov. 12, 2012, 7:15 p.m., Matthias Friedrich wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7858/
> -----------------------------------------------------------
> 
> (Updated Nov. 12, 2012, 7:15 p.m.)
> 
> 
> Review request for crunch.
> 
> 
> Description
> -------
> 
> Here's a patch to test ReviewBoard and to give you an impression of the new Aggregators
class and the changes this would require. I have simplified things even further than I originally
planned, so I need some feedback if I'm on the right track here.
> 
> Even when approved, I will not commit this immediately. There are a few things that need
to be done that I'll discuss on crunch-dev.
> 
> 
> Diffs
> -----
> 
>   crunch-contrib/src/main/java/org/apache/crunch/contrib/bloomfilter/BloomFilterFactory.java
825b445 
>   crunch-examples/src/main/java/org/apache/crunch/examples/AverageBytesByIP.java 8abbb73

>   crunch-examples/src/main/java/org/apache/crunch/examples/TotalBytesByIP.java 44776ea

>   crunch-examples/src/main/java/org/apache/crunch/examples/WordAggregationHBase.java
691721d 
>   crunch/src/it/java/org/apache/crunch/CollectionsIT.java 0d5803e 
>   crunch/src/it/java/org/apache/crunch/WordCountIT.java 5124663 
>   crunch/src/it/java/org/apache/crunch/fn/AggregatorsIT.java PRE-CREATION 
>   crunch/src/it/java/org/apache/crunch/lib/CogroupIT.java b6f5029 
>   crunch/src/it/java/org/apache/crunch/test/Tests.java PRE-CREATION 
>   crunch/src/it/resources/org/apache/crunch/fn/AggregatorsITData/ints.txt PRE-CREATION

>   crunch/src/main/java/org/apache/crunch/Aggregator.java PRE-CREATION 
>   crunch/src/main/java/org/apache/crunch/CombineFn.java d45940b 
>   crunch/src/main/java/org/apache/crunch/PGroupedTable.java e727b70 
>   crunch/src/main/java/org/apache/crunch/fn/Aggregators.java PRE-CREATION 
>   crunch/src/main/java/org/apache/crunch/impl/mem/collect/MemGroupedTable.java 0ee4c3f

>   crunch/src/main/java/org/apache/crunch/impl/mr/collect/PGroupedTableImpl.java fee381d

>   crunch/src/main/java/org/apache/crunch/lib/Aggregate.java f28cca4 
>   crunch/src/test/java/org/apache/crunch/fn/AggregatorsTest.java PRE-CREATION 
>   pom.xml 84d481f 
> 
> Diff: https://reviews.apache.org/r/7858/diff/
> 
> 
> Testing
> -------
> 
> Integration test works, as did a simple example on a single node cluster. Added an integration
test and unit tests based on CombineFnTest.
> 
> 
> Thanks,
> 
> Matthias Friedrich
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message