kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jun Rao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1665) controller state gets stuck in message after execute
Date Thu, 02 Oct 2014 14:46:33 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156627#comment-14156627
] 

Jun Rao commented on KAFKA-1665:
--------------------------------

The reassign partition process only completes after the new replicas are fully caught up and
the old replicas are deleted. So, if the old replica is down, the process can never complete,
which is what you observed. In your case, if you just want to replace a broker host with a
new one, instead of using the reassign partition tool, simply start a new broker with the
same broker id as the old one, the new broker will replicate all the data automatically.

> controller state gets stuck in message after execute
> ----------------------------------------------------
>
>                 Key: KAFKA-1665
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1665
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Joe Stein
>             Fix For: 0.8.3
>
>
> I had a 0.8.1.1 Kafka Broker go down, and I was trying to use the reassign partition
script to move topics off that broker. When I describe the topics, I see the following:
> Topic: mini__022____active_120__33__mini Partition: 0 Leader: 2131118 Replicas: 2131118,2166601,2163421
Isr: 2131118,2166601
> This shows that the broker “2163421” is down. So I create the following file /tmp/move_topic.json:
> {
>     "version": 1,
>     "partitions": [
>         {
>             "topic": "mini__022____active_120__33__mini",
>             "partition": 0,
>             "replicas": [
>                 2131118, 2166601,  2156998
>             ]
>         }
>     ]
> }
> And then do this:
> ./kafka-reassign-partitions.sh --execute --reassignment-json-file /tmp/move_topic.json
> Successfully started reassignment of partitions {"version":1,"partitions":[{"topic":"mini__022____active_120__33__mini","partition":0,"replicas":[2131118,2166601,2156998]}]}
> However, when I try to verify this, I get the following error:
> ./kafka-reassign-partitions.sh --verify --reassignment-json-file /tmp/move_topic.json
> Status of partition reassignment:
> ERROR: Assigned replicas (2131118,2166601,2156998,2163421) don't match the list of replicas
for reassignment (2131118,2166601,2156998) for partition [mini__022____active_120__33__mini,0]
> Reassignment of partition [mini__022____active_120__33__mini,0] failed
> If I describe the topics, I now see there are 4 replicas. This has been like this for
many hours now, so it seems to have permanently moved to 4 replicas for some reason.
> Topic:mini__022____active_120__33__mini PartitionCount:1 ReplicationFactor:4 Configs:
> Topic: mini__022____active_120__33__mini Partition: 0 Leader: 2131118 Replicas: 2131118,2166601,2156998,2163421
Isr: 2131118,2166601
> If I re-execute and re-verify, I get the same error. So it seems to be wedged.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message