flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eron Wright (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5018) Make source idle timeout user configurable
Date Mon, 22 Jan 2018 18:17:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16334670#comment-16334670

Eron Wright  commented on FLINK-5018:

Yes, the above does seem like an unsupported combination.   The Kafka consumer is clearly
a watermark-aware source, and should use the existing idleness functionality (possibly with
some new support for timeouts provided by the source context).  The app should not also make
use of generic idleness logic in this scenario.

> Make source idle timeout user configurable
> ------------------------------------------
>                 Key: FLINK-5018
>                 URL: https://issues.apache.org/jira/browse/FLINK-5018
>             Project: Flink
>          Issue Type: Sub-task
>          Components: DataStream API
>            Reporter: Tzu-Li (Gordon) Tai
>            Priority: Major
>             Fix For: 1.5.0
> There are 2 cases where sources are considered idle and should emit an idle {{StreamStatus}}
downstream, taking Kafka consumer as example:
> - The source instance was not assigned any partitions
> - The source instance was assigned partitions, but they currently don't have any data.
> For the second case, we can only consider it idle after a timeout threshold. It would
be good to make this timeout user configurable besides a default value.

This message was sent by Atlassian JIRA

View raw message