incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills" <jwi...@cloudera.com>
Subject Review Request: Add Aggregators that can operate over Collections and Maps
Date Thu, 13 Dec 2012 22:13:25 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8577/
-----------------------------------------------------------

Review request for crunch and Matthias Friedrich.


Description
-------

Adds support for Aggregators that can be used as part of combineValues ops that include collections
and maps. The cost of the approach is two new methods that were needed on the Aggregator interface:
copy(), to create new instances of a given Aggregator (e.g., one for each new key that is
put into the Map), and arity(), which indicates how many values will be in the Iterable returned
by the results() method if known ahead of time.


This addresses bug CRUNCH-133.
    https://issues.apache.org/jira/browse/CRUNCH-133


Diffs
-----

  crunch-contrib/src/main/java/org/apache/crunch/contrib/bloomfilter/BloomFilterFactory.java
9191a6c 
  crunch/src/main/java/org/apache/crunch/Aggregator.java 432452b 
  crunch/src/main/java/org/apache/crunch/fn/Aggregators.java 0ac79e2 
  crunch/src/test/java/org/apache/crunch/fn/AggregatorsTest.java 6ee1972 

Diff: https://reviews.apache.org/r/8577/diff/


Testing
-------

Unit tests on the collections and maps types included.


Thanks,

Josh Wills


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message