Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kafka.apache.org
Date: Wed, 25 May 2016 22:13:12 +0000 (UTC)
From: "Guozhang Wang (JIRA)" <jira@apache.org>
To: dev@kafka.apache.org
Message-ID: <JIRA.12909504.1446405749000.295446.1464214392851@Atlassian.JIRA>
In-Reply-To: <JIRA.12909504.1446405749000@Atlassian.JIRA>
References: <JIRA.12909504.1446405749000@Atlassian.JIRA> <JIRA.12909504.1446405749131@arcas>
Subject: [jira] [Updated] (KAFKA-2720) Periodic purging groups in the
 coordinator
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 25 May 2016 22:55:09 -0000


     [ https://issues.apache.org/jira/browse/KAFKA-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Guozhang Wang updated KAFKA-2720:
---------------------------------
    Description: 
Currently the coordinator removes the group (i.e. both removing it from the cache and writing the tombstone message on its local replica without waiting for ack) once it becomes an empty group.

This can lead to a few issues such as 1) group removal and creation churns when a group with very few members are being rebalanced, 2) if the local write is failed / not propagated to other followers, they can only be removed again when a new coordinator is migrated and detects the group has no members already.

We could instead piggy-back the periodic offset expiration along with the group purging as well which removes any groups that had no corresponding offsets in the cache any more.

  was:
Currently the coordinator removes the group (i.e. both removing it from the cache and writing the tombstone message on its local replica without waiting for ack) once it becomes an empty group.

This can lead to a few issues such as 1) group removal and creation churns when a group with very few members are being rebalanced, 2) if the local write is failed / not propagated to other followers, they can only be removed again when a new coordinator is migrated and detects the group has no members already.

We could instead piggy-back the periodic offset expiration along with the group purging as well which removes any groups that had no members already.


> Periodic purging groups in the coordinator
> ------------------------------------------
>
>                 Key: KAFKA-2720
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2720
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: consumer
>            Reporter: Guozhang Wang
>            Assignee: Jason Gustafson
>             Fix For: 0.10.1.0
>
>
> Currently the coordinator removes the group (i.e. both removing it from the cache and writing the tombstone message on its local replica without waiting for ack) once it becomes an empty group.
> This can lead to a few issues such as 1) group removal and creation churns when a group with very few members are being rebalanced, 2) if the local write is failed / not propagated to other followers, they can only be removed again when a new coordinator is migrated and detects the group has no members already.
> We could instead piggy-back the periodic offset expiration along with the group purging as well which removes any groups that had no corresponding offsets in the cache any more.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)