flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1870) Reintroduce Indexed reader functionality to streaming
Date Fri, 15 Jan 2016 10:24:39 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101584#comment-15101584
] 

ASF GitHub Bot commented on FLINK-1870:
---------------------------------------

Github user aljoscha commented on the pull request:

    https://github.com/apache/flink/pull/1483#issuecomment-171926900
  
    Hi,
    I would be strongly against exposing the input channel index in any way to operators/user
functions. How the index of a channel maps to an upstream operator is internal to the network
layer and dependent on several things, some of which are not even deterministic IMHO. Off
the top of my head it should depend on the number of upstream tasks, the parallelism of these
upstream tasks and the order in which the system choses to attach the channels.
    
    I think your goal can be achieved by adding a task-id field in the message wrapper and
setting this when emitting elements. Setting this for every processed element (if using the
Storm combat layer or not) seems quite wasteful to me.


> Reintroduce Indexed reader functionality to streaming
> -----------------------------------------------------
>
>                 Key: FLINK-1870
>                 URL: https://issues.apache.org/jira/browse/FLINK-1870
>             Project: Flink
>          Issue Type: Task
>          Components: Streaming
>            Reporter: Gyula Fora
>            Assignee: Matthias J. Sax
>            Priority: Minor
>
> The Indexed record reader classes (IndexedReaderIterator, IndexedMutableReader) were
introduced to allow the streaming operators to access the index of the last read channel from
the input gate. This was a necessary step toward future input sorting operators.
> Unfortunately this untested feature was patched away by the following commit:
> https://github.com/apache/flink/commit/5232c56b13b7e13e2cf10dbe818c221f5557426d
> At some point we need to reimplement these features with proper tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message