gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GEARPUMP-23) Add DSL window (time series) support
Date Tue, 27 Sep 2016 22:21:20 GMT

    [ https://issues.apache.org/jira/browse/GEARPUMP-23?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527582#comment-15527582
] 

ASF GitHub Bot commented on GEARPUMP-23:
----------------------------------------

Github user manuzhang commented on a diff in the pull request:

    https://github.com/apache/incubator-gearpump/pull/85#discussion_r80808708
  
    --- Diff: streaming/src/main/scala/org/apache/gearpump/streaming/dsl/Stream.scala ---
    @@ -115,20 +121,17 @@ class Stream[T](
        *
        * For example,
        * {{{
    -   * Stream[People].groupBy(_.gender).flatmap(..).filter.(..).reduce(..)
    +   * Stream[People].groupBy(_.gender).flatMap(..).filter(..).reduce(..)
        * }}}
        *
    -   * @param fun  Group by function
    +   * @param fn  Group by function
        * @param parallelism  Parallelism level
        * @param description  The description
        * @return  the grouped stream
        */
    -  def groupBy[Group](fun: T => Group, parallelism: Int = 1, description: String =
null)
    -    : Stream[T] = {
    -    val groupOp = GroupByOp(fun, parallelism, Option(description).getOrElse("groupBy"))
    -    graph.addVertex(groupOp)
    -    graph.addEdge(thisNode, edge.getOrElse(Shuffle), groupOp)
    -    new Stream[T](graph, groupOp)
    +  def groupBy[GROUP](fn: T => GROUP, parallelism: Int = 1, description: String = "groupBy")
    +    : GroupByStream[T, GROUP] = {
    +    groupByWindow(DefaultGroupBy(fn), parallelism, description)
    --- End diff --
    
    Yes, I have a new version that does so. Will update today


> Add DSL window (time series) support 
> -------------------------------------
>
>                 Key: GEARPUMP-23
>                 URL: https://issues.apache.org/jira/browse/GEARPUMP-23
>             Project: Apache Gearpump
>          Issue Type: Sub-task
>          Components: streaming
>    Affects Versions: 0.8.0
>            Reporter: Kam Kasravi
>            Assignee: Manu Zhang
>
> Time series support is need for both GEARPUMP-21 and GEARPUMP-22. This was originally
github issue  [1917|https://github.com/gearpump/gearpump/issues/1917]. Other realtime streaming
engines support time windows such as [Flink|https://flink.apache.org/news/2015/12/04/Introducing-windows.html],
[Beam|https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101] and a future release
of [Spark|https://issues.apache.org/jira/secure/attachment/12793410/StructuredStreamingProgrammingAbstractionSemanticsandAPIs-ApacheJIRA.pdf].
A comparison matrix is found [here|https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit#gid=995660187]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message