crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabriel Reid (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-503) Behavior of MAX_N Aggregator for duplicate values is counter-intuitive
Date Thu, 19 Mar 2015 19:20:38 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14369947#comment-14369947
] 

Gabriel Reid commented on CRUNCH-503:
-------------------------------------

I agree that [3,3,2] is the correct answer -- in fact, this is the only correct answer from
my point of view. I would say that if de-duplication is desired then it should be done before
using this aggregator. 

That being said, allowing to specify this as an option on this aggregator would provide a
pretty big potential speedup, as otherwise you'd need to do two reduces in order to de-duplicate
and then get the top-n.

> Behavior of MAX_N Aggregator for duplicate values is counter-intuitive
> ----------------------------------------------------------------------
>
>                 Key: CRUNCH-503
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-503
>             Project: Crunch
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.11.0
>            Reporter: Tycho Lamerigts
>            Assignee: Josh Wills
>
> I would expect code below to return \{1, 2, 3\}. Instead, it returns \{2, 3\}.
> {code}
> public class MaxNAggregatorTest {
>     @Test
>     public void duplicateMaxNValueShouldBeIgnored() {
>         Aggregator<Integer> myAggregator = Aggregators.MAX_N(3, Integer.class);
>         myAggregator.reset();
>         myAggregator.update(1);
>         myAggregator.update(2);
>         myAggregator.update(3);
>         myAggregator.update(3);
>         assertEquals(3, Iterables.size(myAggregator.results()));
>     }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message