kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-2720) Periodic purging groups in the coordinator
Date Wed, 25 May 2016 22:13:12 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Guozhang Wang updated KAFKA-2720:
---------------------------------
    Description: 
Currently the coordinator removes the group (i.e. both removing it from the cache and writing
the tombstone message on its local replica without waiting for ack) once it becomes an empty
group.

This can lead to a few issues such as 1) group removal and creation churns when a group with
very few members are being rebalanced, 2) if the local write is failed / not propagated to
other followers, they can only be removed again when a new coordinator is migrated and detects
the group has no members already.

We could instead piggy-back the periodic offset expiration along with the group purging as
well which removes any groups that had no corresponding offsets in the cache any more.

  was:
Currently the coordinator removes the group (i.e. both removing it from the cache and writing
the tombstone message on its local replica without waiting for ack) once it becomes an empty
group.

This can lead to a few issues such as 1) group removal and creation churns when a group with
very few members are being rebalanced, 2) if the local write is failed / not propagated to
other followers, they can only be removed again when a new coordinator is migrated and detects
the group has no members already.

We could instead piggy-back the periodic offset expiration along with the group purging as
well which removes any groups that had no members already.


> Periodic purging groups in the coordinator
> ------------------------------------------
>
>                 Key: KAFKA-2720
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2720
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: consumer
>            Reporter: Guozhang Wang
>            Assignee: Jason Gustafson
>             Fix For: 0.10.1.0
>
>
> Currently the coordinator removes the group (i.e. both removing it from the cache and
writing the tombstone message on its local replica without waiting for ack) once it becomes
an empty group.
> This can lead to a few issues such as 1) group removal and creation churns when a group
with very few members are being rebalanced, 2) if the local write is failed / not propagated
to other followers, they can only be removed again when a new coordinator is migrated and
detects the group has no members already.
> We could instead piggy-back the periodic offset expiration along with the group purging
as well which removes any groups that had no corresponding offsets in the cache any more.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message