gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GEARPUMP-317) Task minClock could be non increasing
Date Sun, 18 Jun 2017 11:05:02 GMT

    [ https://issues.apache.org/jira/browse/GEARPUMP-317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053163#comment-16053163
] 

ASF GitHub Bot commented on GEARPUMP-317:
-----------------------------------------

Github user huafengw commented on a diff in the pull request:

    https://github.com/apache/incubator-gearpump/pull/187#discussion_r122588208
  
    --- Diff: streaming/src/main/scala/org/apache/gearpump/streaming/dsl/task/GroupByTask.scala
---
    @@ -61,13 +62,14 @@ class GroupByTask[IN, GROUP, OUT](
       }
     
       override def onWatermarkProgress(watermark: Instant): Unit = {
    -    groups.values.forEach(new Consumer[WindowRunner[IN, OUT]] {
    -      override def accept(runner: WindowRunner[IN, OUT]): Unit = {
    -        runner.trigger(watermark).foreach {
    -          result =>
    -            taskContext.output(Message(result.value, result.timestamp))
    +    if (groups.isEmpty && watermark == Watermark.MAX) {
    --- End diff --
    
    Why we have a special case here?


> Task minClock could be non increasing
> -------------------------------------
>
>                 Key: GEARPUMP-317
>                 URL: https://issues.apache.org/jira/browse/GEARPUMP-317
>             Project: Apache Gearpump
>          Issue Type: Bug
>          Components: streaming
>    Affects Versions: 0.8.3
>            Reporter: Manu Zhang
>            Assignee: Manu Zhang
>
> Task minClock is the minimum of upstream minClock and output minClock. Considering the
following scenario,
> 1. Output minClock is set to Long.MaxValue since all messages have been acked by downstream
tasks
> 2. But there are messages buffered in a window function
> 3. Later the window is triggered and those messages are emitted, which will set the output
minClock to a smaller value.
> 4. Plus, the output messages are possibly not in time order, which again will make output
minClock non increasing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message