apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vlad Rozov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (APEXCORE-641) Subscribers/DataListeners may not be scheduled to execute even when they have data to process
Date Tue, 07 Feb 2017 17:48:42 GMT
Vlad Rozov created APEXCORE-641:
-----------------------------------

             Summary: Subscribers/DataListeners may not be scheduled to execute even when
they have data to process
                 Key: APEXCORE-641
                 URL: https://issues.apache.org/jira/browse/APEXCORE-641
             Project: Apache Apex Core
          Issue Type: Bug
          Components: Buffer Server
    Affects Versions: 3.5.0, 3.4.0, 3.2.1, 3.3.0, 3.2.0, 3.6.0
            Reporter: Vlad Rozov
            Assignee: Vlad Rozov


Buffer server iterates over DataListeners aka LogicalNodes and each LogicalNode tries to send
to it's downstream all data that Publisher added to the DataList. When an output port is connected
to multiple partitions or downstream operators (2 or more DataListeners/LogicalNodes) there
may be more data published to the DataList after first few DataListeners in the listeners
set iterated over DataList and reached the last block published so far. The data published
while the last DataListeners sends data to it's downstream will not be processed by other
DataListeners until Publisher adds more data to the DataList. This may lead to blocked operators,
as Buffer server may stop processing data completely in case Publisher fills more than one
Data block while a single DataListener sends data to it's downstream and there are more Subscribers/DataListeners
than number of in memory blocks allowed (8). In such case, Publisher will be suspended, and
there will be no task scheduled to process data already published to the DataList.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message