incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahul Sharma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-74) Reset method of Agregator should have a Configuration param
Date Tue, 25 Sep 2012 03:51:08 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462416#comment-13462416
] 

Rahul Sharma commented on CRUNCH-74:
------------------------------------

The reset method is called for every key for which aggregation is being done by the CombineFn.
Resetting the aggregator for every key may require certain values from configuration, which
could be facilitated this way.  
In aggregator we do not have initialize method as it gets reset every time a new key pops-up.
But in-case we do have such a a method then I think it would be called when the CombineFn
gets called. If I look at this way then I feel this will be a kind of redundant call, at-most
it can hold back the reference to config element but the same is available in CombineFn and
can be directly passed to the reset method.
                
> Reset method of Agregator should have a Configuration param
> -----------------------------------------------------------
>
>                 Key: CRUNCH-74
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-74
>             Project: Crunch
>          Issue Type: New Feature
>            Reporter: Rahul Sharma
>            Priority: Minor
>         Attachments: 0001-Adding-configuration-to-reset-method-in-Aggregator.patch
>
>
> The rest method of aggregator is called for every key that is being received at the reducer.
The purpose of the reset method is to re-initialize any values if required. The initialization
may require some job params or user params from configuration object. There should be a parameter
to reset method which will have the configuration that is being used.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message