kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6583) Metadata should include number of state stores for task
Date Sat, 24 Feb 2018 04:14:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16375295#comment-16375295

Richard Yu commented on KAFKA-6583:

[~mjsax] When updating the rebalance protocol's metadata, is {{SubscriptionInfo}} the only
class that is changed? Because from my knowledge, their are also other variable instances
in {{StreamPartitionAssignor}} which are called "metadata". (e.g. variable of type {{Cluster}}
in {{StreamPartitionAssignor#assign(}}{{Cluster metadata, Map<String, Subscription>
subscriptions}}{{)}})  It appears that only {{SubscriptionInfo}} contains any data which
is useful for assignment, but I just need to confirm that updating metadata would not involve
any other classes.

> Metadata should include number of state stores for task
> -------------------------------------------------------
>                 Key: KAFKA-6583
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6583
>             Project: Kafka
>          Issue Type: New Feature
>          Components: streams
>    Affects Versions:,
>            Reporter: Richard Yu
>            Priority: Critical
>              Labels: needs-kip
> Currently, in the need for clients to be more evenly balanced, stateful tasks should
be distributed in such a manner that it will be spread equally. However, for such an awareness
to be implemented during task assignment, it would require the need for the present rebalance
protocol metadata to also contain the number of state stores in a particular task. This way,
it will allow us to "weight" tasks during assignment. 

This message was sent by Atlassian JIRA

View raw message