apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXCORE-494) Window id of downstream operator is not moving after dynamic partition of upstream operator.
Date Thu, 21 Jul 2016 16:28:21 GMT

    [ https://issues.apache.org/jira/browse/APEXCORE-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387994#comment-15387994
] 

ASF GitHub Bot commented on APEXCORE-494:
-----------------------------------------

Github user chaithu14 commented on a diff in the pull request:

    https://github.com/apache/apex-core/pull/359#discussion_r71739128
  
    --- Diff: engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java ---
    @@ -895,8 +895,10 @@ private void redoPartitions(PMapping currentMapping, String note)
             addedPartitions.add(newPartition);
           } else {
             // check whether mapping was changed
    +        int currentPartitionsSize = mainPC.currentPartitions.size();
             for (DefaultPartition<Operator> pi : mainPC.currentPartitions) {
    -          if (pi == newPartition && pi.isModified()) {
    +          if (pi == newPartition && (pi.isModified() ||
    --- End diff --
    
    @vrozov The problem occurs when definPartitions() returns the list of partitions (size
> 1)  which consists of current partition of size 1.  Unifier will come into the picture
in newPartitions  which is not be there in initial launch. Here, port mapping of current partition
which is in mainPC.newPartitions has to be changed.


> Window id of downstream operator is not moving after dynamic partition of upstream operator.
> --------------------------------------------------------------------------------------------
>
>                 Key: APEXCORE-494
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-494
>             Project: Apache Apex Core
>          Issue Type: Bug
>            Reporter: Chaitanya
>            Assignee: Chaitanya
>         Attachments: PhysicalDAGAfterKafkaRepartition.png, PhysicalDAGBeforeKafkaRepartition.png,
PhysicalOperatorAfterKafkaRepartition.png
>
>
> Created the following application to reproduce this issue.
> Application has the following DAG:
>    KafkaSinglePortStringInputOperator(Input) -> ConsoleOutputOperator(Output)
>    
>    I launched the application with below configuration:
> Kafka topic created with single partition and replication factor as 1.
> Partition Strategy: ONE_TO_ONE
>    Launched the application successfully. After some time, I increased the topic partitions
to 2. After re-partition, the window of down stream operator is not moving. By looking into
the app Physical DAG, it looks like there is an issue in construction of Physical DAG after
re-partition.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message