flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2398) Decouple StreamGraph Building from the API
Date Wed, 05 Aug 2015 09:13:05 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14655052#comment-14655052
] 

ASF GitHub Bot commented on FLINK-2398:
---------------------------------------

Github user gyfora commented on the pull request:

    https://github.com/apache/flink/pull/988#issuecomment-127926607
  
    If I understand correctly, this also this changes the semantics that we execute programs
without sinks, and also topology branches which don't end in sinks. I personally don't like
the fact that the each branch in the processing graph needs to end in a sink, it is rather
artificial.


> Decouple StreamGraph Building from the API
> ------------------------------------------
>
>                 Key: FLINK-2398
>                 URL: https://issues.apache.org/jira/browse/FLINK-2398
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming
>            Reporter: Aljoscha Krettek
>            Assignee: Aljoscha Krettek
>
> Currently, the building of the StreamGraph is very intertwined with the API methods.
DataStream knows about the StreamGraph and keeps track of splitting, selected names, unions
and so on. This leads to the problem that is is very hard to understand how the StreamGraph
is built because the code that does it is all over the place. This also makes it hard to extend/change
parts of the Streaming system.
> I propose to introduce "Transformations". A transformation hold information about one
operation: The input streams, types, names, operator and so on. An API method creates a transformation
instead of fiddling with the StreamGraph directly. A new component, the StreamGraphGenerator
creates a StreamGraph from the tree of transformations that result from program specification
using the API methods. This would relieve DataStream from knowing about the StreamGraph and
makes unions, splitting, selection visible transformations instead of being scattered across
the different API classes as fields.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message