apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "bright chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2076) AbstractExactlyOnceKafkaOutputOperator didn't handle the orderless of tuples in a window
Date Tue, 10 May 2016 18:53:12 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278676#comment-15278676
] 

bright chen commented on APEXMALHAR-2076:
-----------------------------------------

2. Problems with partitions: assume there are multiple operator partitions and multiple Kafka
partitions
1) Kafka partition stick to the operator partition. This mechanism will cause problem if the
upstream operator crash, the data which sent to operator partition 1 could send to partition
2 when replay.

2) Kafka partition stick to the data. This means there could have multiple operator partition
write to the same Kafka partition. This will cause problem if the one operator partition crash.
As when this partition recovery, a lot of data could already written to the Kafka partition
by another operator partition 

> AbstractExactlyOnceKafkaOutputOperator didn't handle the orderless of tuples in a window
> ----------------------------------------------------------------------------------------
>
>                 Key: APEXMALHAR-2076
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2076
>             Project: Apache Apex Malhar
>          Issue Type: Bug
>            Reporter: bright chen
>            Assignee: bright chen
>
> The order of the tuples in the same window are not guaranteed in replay. AbstractExactlyOnceKafkaOutputOperator's
logic assume the replayed tuples have same order.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message