kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clemens Valiente (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-6391) output from ensure copartitioning is not used for Cluster metadata
Date Wed, 20 Dec 2017 15:40:00 GMT
Clemens Valiente created KAFKA-6391:
---------------------------------------

             Summary: output from ensure copartitioning is not used for Cluster metadata
                 Key: KAFKA-6391
                 URL: https://issues.apache.org/jira/browse/KAFKA-6391
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 1.0.0
            Reporter: Clemens Valiente
            Assignee: Clemens Valiente


https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamPartitionAssignor.java#L366


Map<TopicPartition, PartitionInfo> allRepartitionTopicPartitions is created from repartitionTopicMetadata
THEN we do ensureCoPartitioning on repartitionTopicMetadata
THEN we create topics and partitions according to repartitionTopicMetadata
THEN we use allRepartitionTopicPartitions to create our Cluster fullMetadata
THEN we use fullMetadata to assign the tasks and no longer use repartitionTopicMetadata

This results in any change to repartitionTopicMetadata in ensureCoPartitioning to be used
for creating partitions but no tasks are ever created for any partition added by ensureCoPartitioning()


the fix is easy: First ensureCoPartitioning() on repartitionTopicMetadata before creating
allRepartitionTopicPartitions




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message