kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sophie Blee-Goldman (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-9020) Streams sub-topologies should be sorted by sink -> source relationship
Date Mon, 14 Oct 2019 20:20:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sophie Blee-Goldman resolved KAFKA-9020.
    Fix Version/s: 2.4.0
       Resolution: Fixed

> Streams sub-topologies should be sorted by sink -> source relationship
> ----------------------------------------------------------------------
>                 Key: KAFKA-9020
>                 URL: https://issues.apache.org/jira/browse/KAFKA-9020
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Guozhang Wang
>            Assignee: Sophie Blee-Goldman
>            Priority: Major
>             Fix For: 2.4.0
> In Streams {{InternalTopologyBuilder}} we are sorting the sub-topologies (named as node
groups) based on the source node names:
> {code}
> for (final String nodeName : Utils.sorted(allSourceNodes)) {
>             nodeGroupId = putNodeGroupName(nodeName, nodeGroupId, nodeGroups, rootToNodeGroup);
>         }
> {code}
> This is okay for DSL where the source node names are not customized, and hence we can
just rely on the internal suffix {{..-SOURCE-xxxxx}} which is basically following the parsing
order. But now with the source node name customized with KIP-307, we should no longer rely
on the source node names. Instead, we should sort sub-topologies by the sink -> source
relationship --- that is, if a sub-topology A has a sink node sending to topic X while another
sub-topology B has a source node reading from X, then A should be sorted before B.

This message was sent by Atlassian Jira

View raw message