apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pramod Immaneni (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXCORE-570) Prevent upstream operators from getting too far ahead when downstream operators are slow
Date Wed, 02 Nov 2016 20:06:58 GMT

    [ https://issues.apache.org/jira/browse/APEXCORE-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630269#comment-15630269
] 

Pramod Immaneni commented on APEXCORE-570:
------------------------------------------

For the inter-process case, if the downstream operator is slower than the speed of spooling
then it causes the upstream to pull ahead. This is the scenario I am facing and working on
addressing. It cannot be controlled with an absolute capacity limit (memory+spool) because
of fault tolerance older windows are needed till committed, it would have to be a window difference,
how much ahead the upstream operator is allowed to go from the minimum of all downstream operators
before breaks are applied on its publishing.

> Prevent upstream operators from getting too far ahead when downstream operators are slow
> ----------------------------------------------------------------------------------------
>
>                 Key: APEXCORE-570
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-570
>             Project: Apache Apex Core
>          Issue Type: Improvement
>            Reporter: Pramod Immaneni
>            Assignee: Pramod Immaneni
>
> If the downstream operators are slower than upstream operators then the upstream operators
will get ahead and the gap can continue to increase. Provide an option to slow down or temporarily
pause the upstream operators when they get too far ahead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message