gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (GEARPUMP-249) reduce is not executed when following flatMap in Stream DSL
Date Sun, 11 Dec 2016 02:53:58 GMT
Manu Zhang created GEARPUMP-249:
-----------------------------------

             Summary: reduce is not executed when following flatMap in Stream DSL
                 Key: GEARPUMP-249
                 URL: https://issues.apache.org/jira/browse/GEARPUMP-249
             Project: Apache Gearpump
          Issue Type: Bug
          Components: streaming
    Affects Versions: 0.8.2
            Reporter: Manu Zhang
            Assignee: Manu Zhang


The issue can be reproduced if a redundant {{map}} is added between {{groupBy}} and {{sum}}

{code}
    app.source(data.lines.toList, 1, "source").
      // word => (word, count)
      flatMap(line => line.split("[\\s]+")).map((_, 1)).
      // (word, count1), (word, count2) => (word, count1 + count2)
      groupByKey().sum.log
{code}

The problem is with {{AndThen#process}} method, where {{flatMap}} is lazily/ never executed.

{code}
  override def process(value: IN): TraversableOnce[OUT] = {
    first.process(value).flatMap(second.process)
  }
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message