crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tycho Lamerigts (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-503) Behavior of MAX_N Aggregator for duplicate values is counter-intuitive
Date Thu, 19 Mar 2015 19:43:38 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14369981#comment-14369981
] 

Tycho Lamerigts commented on CRUNCH-503:
----------------------------------------

I think a case can be made for both answers. If only because my current use case requires
\{3, 2, 1\}. :-) So an option to choose duplication behaviour would be ideal for me.

I currently de-duplicate by wrapping the MAX_N Aggregator in a delegating aggregator that
does something like this

{code}
if (!Iterables.contains(this.delegate.result(), value) {
   this.delegate.update(value)
}
{code} 

Not pretty, but it does avoid another MR step.

> Behavior of MAX_N Aggregator for duplicate values is counter-intuitive
> ----------------------------------------------------------------------
>
>                 Key: CRUNCH-503
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-503
>             Project: Crunch
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.11.0
>            Reporter: Tycho Lamerigts
>            Assignee: Josh Wills
>
> I would expect code below to return \{1, 2, 3\}. Instead, it returns \{2, 3\}.
> {code}
> public class MaxNAggregatorTest {
>     @Test
>     public void duplicateMaxNValueShouldBeIgnored() {
>         Aggregator<Integer> myAggregator = Aggregators.MAX_N(3, Integer.class);
>         myAggregator.reset();
>         myAggregator.update(1);
>         myAggregator.update(2);
>         myAggregator.update(3);
>         myAggregator.update(3);
>         assertEquals(3, Iterables.size(myAggregator.results()));
>     }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message