gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GEARPUMP-23) Add DSL window (time series) support
Date Wed, 28 Sep 2016 01:04:20 GMT

    [ https://issues.apache.org/jira/browse/GEARPUMP-23?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527971#comment-15527971
] 

ASF GitHub Bot commented on GEARPUMP-23:
----------------------------------------

Github user manuzhang commented on the issue:

    https://github.com/apache/incubator-gearpump/pull/85
  
    @kkasravi 
    
    `stream.groupBy` is now a shortcut for `stream.window(CountWindow.apply(1).triggering(CountTrigger).accumulating).groupBy`.
    
    so `groupBy` has windowing semantics and all window operations should follow `groupBy`
    
    here is the window semantics
    
    ```
    - Window
        - WindowFn 
             -> SlidingWindowFn // FixedWindow and SlidingWindow 
             -> CountWindowFn  // CountWindow
        - Trigger
             -> EventTimeTrigger  // event time aggregation
             -> ProcessingTimeTrigger  // processing time aggregation
             -> CountTrigger // count aggregation, used with CountWindowFn
         - AccumulationMode
             -> Accumulating   // states are accumulated across windows
             -> Discarding  // states are not accumulated  
    ```


> Add DSL window (time series) support 
> -------------------------------------
>
>                 Key: GEARPUMP-23
>                 URL: https://issues.apache.org/jira/browse/GEARPUMP-23
>             Project: Apache Gearpump
>          Issue Type: Sub-task
>          Components: streaming
>    Affects Versions: 0.8.0
>            Reporter: Kam Kasravi
>            Assignee: Manu Zhang
>
> Time series support is need for both GEARPUMP-21 and GEARPUMP-22. This was originally
github issue  [1917|https://github.com/gearpump/gearpump/issues/1917]. Other realtime streaming
engines support time windows such as [Flink|https://flink.apache.org/news/2015/12/04/Introducing-windows.html],
[Beam|https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101] and a future release
of [Spark|https://issues.apache.org/jira/secure/attachment/12793410/StructuredStreamingProgrammingAbstractionSemanticsandAPIs-ApacheJIRA.pdf].
A comparison matrix is found [here|https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit#gid=995660187]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message