flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-5719) Let LatencyMarkers completely bypass operators / chains
Date Mon, 06 Feb 2017 13:07:41 GMT

     [ https://issues.apache.org/jira/browse/FLINK-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tzu-Li (Gordon) Tai updated FLINK-5719:
---------------------------------------
    Description: 
Currently, {{LatencyMarker}} s are forwarded through operators via the operator interfaces
and methods, i.e. {{AbstractStreamOperator#processLatencyMarker()}},  {{Output#emitLatencyMarker()}},
{{OneInputStreamOperator#processLatencyMarker()}} etc.

The main issue with this is that {{LatencyMarker}} s are essentially internal elements, and
the implementation on how to handle them should be final. Exposing them through operator interfaces
will allow the user to override the implementation, and also makes the user interface for
operators over-complicated.

[~aljoscha] suggested to bypass such internal stream elements from the operator to keep the
operator interfaces minimal, in FLINK-5017.
We propose a similar approach here for {{LatencyMarker}}s as well. Since the chaining output
calls contribute very little to the measured latency and can be ignored, instead of passing
it through operator chains, latency markers can simply be passed downstream once tasks receive
them.

  was:
Currently, {{LatencyMarker}}s are forwarded through operators via the operator interfaces
and methods, i.e. {{AbstractStreamOperator#processLatencyMarker()}},  {{Output#emitLatencyMarker()}},
{{OneInputStreamOperator#processLatencyMarker()}} etc.

The main issue with this is that {{LatencyMarker}}s are essentially internal elements, and
the implementation on how to handle them should be final. Exposing them through operator interfaces
will allow the user to override the implementation, and also makes the user interface for
operators over-complicated.

[~aljoscha] suggested to bypass such internal stream elements from the operator to keep the
operator interfaces minimal, in FLINK-5017.
We propose a similar approach here for {{LatencyMarker}}s as well. Since the chaining output
calls contribute very little to the measured latency and can be ignored, instead of passing
it through operator chains, latency markers can simply be passed downstream once tasks receive
them.


> Let LatencyMarkers completely bypass operators / chains
> -------------------------------------------------------
>
>                 Key: FLINK-5719
>                 URL: https://issues.apache.org/jira/browse/FLINK-5719
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming
>            Reporter: Tzu-Li (Gordon) Tai
>
> Currently, {{LatencyMarker}} s are forwarded through operators via the operator interfaces
and methods, i.e. {{AbstractStreamOperator#processLatencyMarker()}},  {{Output#emitLatencyMarker()}},
{{OneInputStreamOperator#processLatencyMarker()}} etc.
> The main issue with this is that {{LatencyMarker}} s are essentially internal elements,
and the implementation on how to handle them should be final. Exposing them through operator
interfaces will allow the user to override the implementation, and also makes the user interface
for operators over-complicated.
> [~aljoscha] suggested to bypass such internal stream elements from the operator to keep
the operator interfaces minimal, in FLINK-5017.
> We propose a similar approach here for {{LatencyMarker}}s as well. Since the chaining
output calls contribute very little to the measured latency and can be ignored, instead of
passing it through operator chains, latency markers can simply be passed downstream once tasks
receive them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message