Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Date: Thu, 26 May 2016 17:47:12 +0000 (UTC)
From: "Robert Metzger (JIRA)" <jira@apache.org>
To: issues@flink.apache.org
Message-ID: <JIRA.12924007.1450879418000.306061.1464284832951@Atlassian.JIRA>
In-Reply-To: <JIRA.12924007.1450879418000@Atlassian.JIRA>
References: <JIRA.12924007.1450879418000@Atlassian.JIRA> <JIRA.12924007.1450879418729@arcas>
Subject: [jira] [Commented] (FLINK-3190) Retry rate limits for DataStream
 API
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Thu, 26 May 2016 17:47:14 -0000


    [ https://issues.apache.org/jira/browse/FLINK-3190?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1530=
2566#comment-15302566 ]=20

Robert Metzger commented on FLINK-3190:
---------------------------------------

[~packet], since you've requested this feature, would you be available to r=
eview and test the pull request?

> Retry rate limits for DataStream API
> ------------------------------------
>
>                 Key: FLINK-3190
>                 URL: https://issues.apache.org/jira/browse/FLINK-3190
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Sebastian Klemke
>            Assignee: Micha=C5=82 Fijo=C5=82ek
>            Priority: Minor
>
> For a long running stream processing job, absolute numbers of retries don=
't make much sense: The job will accumulate transient errors over time and =
will die eventually when thresholds are exceeded. Rate limits are better su=
ited in this scenario: A job should only die, if it fails too often in a gi=
ven time frame. To better overcome transient errors, retry delays could be =
used, as suggested in other issues.
> Absolute numbers of retries can still make sense, if failing operators do=
n't make any progress at all. We can measure progress by OperatorState chan=
ges and by observing output, as long as the operator in question is not a s=
ink. If operator state changes and/or operator produces output, we can assu=
me it makes progress.
> As an example, let's say we configured a retry rate limit of 10 retries p=
er hour and a non-sink operator A. If the operator fails once every 10 minu=
tes and produces output between failures, it should not lead to job termina=
tion. But if the operator fails 11 times in an hour or does not produce out=
put between 11 consecutive failures, job should be terminated.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)