Mailing-List: contact dev-help@gearpump.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@gearpump.incubator.apache.org
Date: Sun, 11 Dec 2016 02:53:58 +0000 (UTC)
From: "Manu Zhang (JIRA)" <jira@apache.org>
To: dev@gearpump.incubator.apache.org
Message-ID: <JIRA.13027235.1481424835000.492346.1481424838325@Atlassian.JIRA>
In-Reply-To: <JIRA.13027235.1481424835000@Atlassian.JIRA>
References: <JIRA.13027235.1481424835000@Atlassian.JIRA> <JIRA.13027235.1481424835433@arcas>
Subject: [jira] [Created] (GEARPUMP-249) reduce is not executed when
 following flatMap in Stream DSL
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Sun, 11 Dec 2016 02:54:03 -0000

Manu Zhang created GEARPUMP-249:
-----------------------------------

             Summary: reduce is not executed when following flatMap in Stream DSL
                 Key: GEARPUMP-249
                 URL: https://issues.apache.org/jira/browse/GEARPUMP-249
             Project: Apache Gearpump
          Issue Type: Bug
          Components: streaming
    Affects Versions: 0.8.2
            Reporter: Manu Zhang
            Assignee: Manu Zhang


The issue can be reproduced if a redundant {{map}} is added between {{groupBy}} and {{sum}}

{code}
    app.source(data.lines.toList, 1, "source").
      // word => (word, count)
      flatMap(line => line.split("[\\s]+")).map((_, 1)).
      // (word, count1), (word, count2) => (word, count1 + count2)
      groupByKey().sum.log
{code}

The problem is with {{AndThen#process}} method, where {{flatMap}} is lazily/ never executed.

{code}
  override def process(value: IN): TraversableOnce[OUT] = {
    first.process(value).flatMap(second.process)
  }
{code}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)