kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Noll (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-4875) Kafka Streams: topic groups and builder.stream API
Date Thu, 09 Mar 2017 12:27:37 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-4875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Michael Noll updated KAFKA-4875:
    Summary: Kafka Streams: topic groups and builder.stream API  (was: Kafka streams topic
groups and builder.stream API)

> Kafka Streams: topic groups and builder.stream API
> --------------------------------------------------
>                 Key: KAFKA-4875
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4875
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>    Affects Versions:
>            Reporter: Eno Thereska
>             Fix For:
> One thing that has come up in mailing list is that the notion of topic groups is unclear.
So if you have 2 topics, each with 3 partitions, you'd expect 6 tasks. However, if you do
`builder.stream(topic1, topic2)` you actually get only 3 tasks created. If you do `builder.stream(topic1);
builder.stream(topic2)` you get 6 tasks, i.e., parallelism is increased. So the same application,
calling builder.stream() in two different ways, might see different performance.
> In the Kafka Streams documentations we mention partitions and tasks, but not topic groups.
We also do not document the effects of using builder.stream with a topic array. We also need
to revisit whether the API and its effects are confusing.

This message was sent by Atlassian JIRA

View raw message