gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GEARPUMP-249) reduce is not executed when following flatMap in Stream DSL
Date Wed, 04 Jan 2017 04:51:58 GMT

    [ https://issues.apache.org/jira/browse/GEARPUMP-249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15797168#comment-15797168
] 

ASF GitHub Bot commented on GEARPUMP-249:
-----------------------------------------

Github user huafengw commented on the issue:

    https://github.com/apache/incubator-gearpump/pull/127
  
    +1


> reduce is not executed when following flatMap in Stream DSL
> -----------------------------------------------------------
>
>                 Key: GEARPUMP-249
>                 URL: https://issues.apache.org/jira/browse/GEARPUMP-249
>             Project: Apache Gearpump
>          Issue Type: Bug
>          Components: streaming
>    Affects Versions: 0.8.2
>            Reporter: Manu Zhang
>            Assignee: Manu Zhang
>             Fix For: 0.8.3
>
>
> The issue can be reproduced if a redundant {{map}} is added between {{groupBy}} and {{sum}}
> {code}
>     app.source(data.lines.toList, 1, "source").
>       // word => (word, count)
>       flatMap(line => line.split("[\\s]+")).map((_, 1)).
>       // (word, count1), (word, count2) => (word, count1 + count2)
>       groupByKey().sum.log
> {code}
> The problem is with {{AndThen#process}} method, where {{flatMap}} is lazily/ never executed.
> {code}
>   override def process(value: IN): TraversableOnce[OUT] = {
>     first.process(value).flatMap(second.process)
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message