kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4696) Streams standby task assignment should be state-store aware
Date Thu, 22 Feb 2018 22:36:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373589#comment-16373589

ASF GitHub Bot commented on KAFKA-4696:

ConcurrencyPractitioner opened a new pull request #4615: [KAFKA-4696] Streams standby task
assignment should be state-store aware
URL: https://github.com/apache/kafka/pull/4615

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

> Streams standby task assignment should be state-store aware
> -----------------------------------------------------------
>                 Key: KAFKA-4696
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4696
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: streams
>    Affects Versions:,
>            Reporter: Damian Guy
>            Priority: Major
> Task Assignment is currently not aware of which tasks have State Stores. This can result
in uneven balance of standby task assignment as all tasks are assigned, but only those tasks
with state-stores are ever created by {{StreamThread}}. So what seems like an optimal strategy
during assignment time could be sub-optimal post-assignment.
> For example, lets say we have 4 tasks (2 with state-stores), 2 clients, numStandbyReplicas
= 1. Each client would get 2 active and 2 standby tasks.  One of the clients may end up with
both state-store tasks, while the other has none.
> Further to this, standby task configuration is currently "all or nothing". It might make
sense to allow more fine grained configuration, i.e., the ability to specify the number of
standby replicas individually for each stateful operator.

This message was sent by Atlassian JIRA

View raw message