gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GEARPUMP-23) Add DSL window (time series) support
Date Tue, 27 Sep 2016 17:29:20 GMT

    [ https://issues.apache.org/jira/browse/GEARPUMP-23?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15526847#comment-15526847
] 

ASF GitHub Bot commented on GEARPUMP-23:
----------------------------------------

Github user kkasravi commented on a diff in the pull request:

    https://github.com/apache/incubator-gearpump/pull/85#discussion_r80745374
  
    --- Diff: streaming/src/main/scala/org/apache/gearpump/streaming/dsl/Stream.scala ---
    @@ -147,6 +150,33 @@ class Stream[T](
         graph.addEdge(thisNode, edge.getOrElse(Shuffle), processorOp)
         new Stream[R](graph, processorOp, Some(Shuffle))
       }
    +
    +  private def groupByWindow[GROUP](fn: GroupByFn[T, GROUP],
    +      parallelism: Int, description: String): GroupByStream[T, GROUP] = {
    +    val groupOp = GroupByOp[T, GROUP](fn,
    +      parallelism, description)
    +    graph.addVertex(groupOp)
    +    graph.addEdge(thisNode, edge.getOrElse(Shuffle), groupOp)
    +    new GroupByStream[T, GROUP](graph, groupOp)
    +  }
    +
    +}
    +
    +class GroupByStream[T, GROUP](
    +    private val graph: Graph[Op, OpEdge],
    +    private val groupByOp: GroupByOp[T, GROUP],
    +    private val edge: Option[OpEdge] = None) extends Stream[T](graph, groupByOp, edge)
{
    --- End diff --
    
    Is 'private val graph' equivalent to 'graph' (val adds public accessors so private takes
them away?)


> Add DSL window (time series) support 
> -------------------------------------
>
>                 Key: GEARPUMP-23
>                 URL: https://issues.apache.org/jira/browse/GEARPUMP-23
>             Project: Apache Gearpump
>          Issue Type: Sub-task
>          Components: streaming
>    Affects Versions: 0.8.0
>            Reporter: Kam Kasravi
>            Assignee: Manu Zhang
>
> Time series support is need for both GEARPUMP-21 and GEARPUMP-22. This was originally
github issue  [1917|https://github.com/gearpump/gearpump/issues/1917]. Other realtime streaming
engines support time windows such as [Flink|https://flink.apache.org/news/2015/12/04/Introducing-windows.html],
[Beam|https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101] and a future release
of [Spark|https://issues.apache.org/jira/secure/attachment/12793410/StructuredStreamingProgrammingAbstractionSemanticsandAPIs-ApacheJIRA.pdf].
A comparison matrix is found [here|https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit#gid=995660187]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message