kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mayuresh Gharat (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-2976) Mirror maker dies if we delete a topic from destination cluster
Date Tue, 23 Aug 2016 18:36:20 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mayuresh Gharat resolved KAFKA-2976.
------------------------------------
    Resolution: Won't Fix

> Mirror maker dies if we delete a topic from destination cluster
> ---------------------------------------------------------------
>
>                 Key: KAFKA-2976
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2976
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients
>            Reporter: Mayuresh Gharat
>            Assignee: Mayuresh Gharat
>
> In datapipeline,
> 1) Suppose the  Mirror Maker is producing to a cluster with Topic T and has 128 partitions
(Partition 0 to Partition 127) . The default setting on creation of a new topic on that cluster
is 8 partitions.
> 2) After we delete the topic, the topic gets recreated with 8 partitions (Partition 0
to Partition 7).
> 3) The RecordAccumulator has batches for partitions from 9 to 127. Those batches get
expired and the mirror makers will die to avoid data loss.
> We need a way to reassign those batches (batches for Partition 9 top Partition 127) in
the RecordAccumulator to the newly created Topic T with 8 partitions (Partition 0 to Partition
7).  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message