crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Commented] (CRUNCH-58) Implement PObject in Crunch/Scrunch
Date Fri, 07 Sep 2012 12:49:08 GMT


Josh Wills commented on CRUNCH-58:

After a good night's sleep, I'm back to my "some abstract function that did a computation
on the materialized iterable" idea. My thought was that we could implement the current materializeAsMap
functionality to return a PObject<Map<K, V>>, where the abstract function would
take the Iterable<Pair<K, V>> and, when it was actually materialized, do the computations
to convert it to a Map instance. PObject could save us from the inevitable creation of materializeAsSet,
materializeAsList, materializeAsMultimap, etc.
> Implement PObject in Crunch/Scrunch
> -----------------------------------
>                 Key: CRUNCH-58
>                 URL:
>             Project: Crunch
>          Issue Type: New Feature
>    Affects Versions: 0.3.0
>            Reporter: Kiyan Ahmadizadeh
>            Assignee: Kiyan Ahmadizadeh
> FlumeJava has the concept of a PObject<T>, a container for a singleton of type
T.  It is meant represent the result of a distributed computation that yields a singleton
value (for example max, min, and length methods on PCollection<T>).  Generally speaking,
the result of any computation that combines/reduces a PCollection into a singleton value could
be represented by a PObject.  
> Like PCollection, a PObject defers distributed computation until its value is actually

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message