beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-956) Execute ReduceFnRunner Directly in Flink Runner
Date Thu, 24 Nov 2016 10:39:58 GMT

    [ https://issues.apache.org/jira/browse/BEAM-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15692922#comment-15692922
] 

ASF GitHub Bot commented on BEAM-956:
-------------------------------------

GitHub user aljoscha opened a pull request:

    https://github.com/apache/incubator-beam/pull/1435

    Replace WindowAssignment OldDoFn by FlatMap in FLink Runner

    The streaming runner had an OldDoFn that was used for assigning windows
    using a WindowFn. This is now done with a FlatMap.
    
    R: @mxm for review please
    CC: @kennknowles I thought you might enjoy seeing `OldDoFns` removed. 😃 
    
    This is the first step towards getting rid of `OldDoFn` in the Flink runner. The next
steps are:
     - https://issues.apache.org/jira/browse/BEAM-956: Execute ReduceFnRunner Directly in
Flink Runner
     - https://issues.apache.org/jira/browse/BEAM-843: Use New DoFn Directly in Flink Runner

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/aljoscha/incubator-beam flink-remove-olddofn

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/1435.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1435
    
----
commit da32dcd06dddc8f4e288e8778fd8ac4a572d72ab
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2016-11-08T10:03:21Z

    Replace WindowAssignment OldDoFn by FlatMap in FLink Runner
    
    The streaming runner had an OldDoFn that was used for assigning windows
    using a WindowFn. This is now done with a FlatMap.

----


> Execute ReduceFnRunner Directly in Flink Runner
> -----------------------------------------------
>
>                 Key: BEAM-956
>                 URL: https://issues.apache.org/jira/browse/BEAM-956
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-flink
>            Reporter: Aljoscha Krettek
>
> Right now, a {{ReduceFnRunner}} is executed via {{GroupAlsoByWindowViaWindowSetDoFn}}
which in turn is executed via a {{DoFnRunner}}. We should change that to get rid of the dependence
on {{GroupAlsoByWindowViaWindowSetDoFn}} which is an {{OldDoFn}} and also to get rid of some
unneeded layering.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message