incubator-kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: no partition leader yet for partition 0 (0.8.0)
Date Wed, 28 Nov 2012 05:13:31 GMT
Is a broker down in your test? If so, you could
see LeaderNotAvailableException in the producer. The producer is trying to
refresh the metadata and the leader may not have been elected yet. You
shouldn't see it often though.

Thanks,

Jun

On Tue, Nov 27, 2012 at 1:11 PM, Chris Curtin <curtin.chris@gmail.com>wrote:

> Hi,
>
> I noticed several errors when writing to a topic with 5 partitions. It
> looks like the data was written to all 3 brokers, but I get the following
> errors:
>
> 9961 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Metadata for topic
> partition [test1, 0] is errornous:
> [PartitionMetadata(0,None,WrappedArray(),WrappedArray(),5)]
> kafka.common.LeaderNotAvailableException
> at sun.reflect.GeneratedConstructorAccessor1.newInstance(Unknown Source)
> at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> <snip>
>
> 9962 [main] DEBUG kafka.producer.async.DefaultEventHandler  - Getting the
> number of broker partitions registered for topic: test1
> 9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Getting broker
> partition info for topic test1
> 9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Topic test1
> partition 0 does not have a leader yet
> 9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Topic test1
> partition 1 has leader 7
> 9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Topic test1
> partition 2 has leader 8
> 9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Topic test1
> partition 3 has leader 9
> 9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Topic test1
> partition 4 has leader 1
> 9963 [main] DEBUG kafka.producer.async.DefaultEventHandler  - Broker
> partitions registered for topic: test1 are 0,1,2,3,4
>
> This happens a lot as I write data to the Broker.
>
> Topic was created with:
>
> -bash-3.2$ ./kafka-create-topic.sh --topic test1 --partition 5 --replica 3
> --zookeeper localhost:2181
>
> Doing a list of topics shows an empty list for that partition:
>
> [2012-11-27 16:03:35,604] INFO Session establishment complete on server
> localhost/127.0.0.1:2181, sessionid = 0x23b4218eccd000b, negotiated
> timeout
> = 30000 (org.apache.zookeeper.ClientCnxn)
> [2012-11-27 16:03:35,607] INFO zookeeper state changed (SyncConnected)
> (org.I0Itec.zkclient.ZkClient)
> topic: test1
> PartitionMetadata(0,None,List(),List(),5)
>
> PartitionMetadata(1,Some(id:7,creatorId:10.121.31.57-1354023708335,host:10.121.31.57,port:9092),List(id:7,creatorId:10.121.31.57-1354023708335,host:10.121.31.57,port:9092,
> id:8,creatorId:10.121.31.57-1354023708340,host:10.121.31.57,port:9093,
>
> id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094),ArrayBuffer(id:7,creatorId:10.121.31.57-1354023708335,host:10.121.31.57,port:9092,
> id:8,creatorId:10.121.31.57-1354023708340,host:10.121.31.57,port:9093,
> id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094),0)
>
> PartitionMetadata(2,Some(id:8,creatorId:10.121.31.57-1354023708340,host:10.121.31.57,port:9093),List(id:8,creatorId:10.121.31.57-1354023708340,host:10.121.31.57,port:9093,
> id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094,
>
> id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092),ArrayBuffer(id:8,creatorId:10.121.31.57-1354023708340,host:10.121.31.57,port:9093,
> id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094,
> id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092),0)
>
> PartitionMetadata(3,Some(id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094),List(id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094,
> id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092,
>
> id:2,creatorId:10.121.31.55-1354023701344,host:10.121.31.55,port:9093),ArrayBuffer(id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094,
> id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092,
> id:2,creatorId:10.121.31.55-1354023701344,host:10.121.31.55,port:9093),0)
>
> PartitionMetadata(4,Some(id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092),List(id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092,
> id:2,creatorId:10.121.31.55-1354023701344,host:10.121.31.55,port:9093,
>
> id:3,creatorId:10.121.31.55-1354023701345,host:10.121.31.55,port:9094),ArrayBuffer(id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092,
> id:2,creatorId:10.121.31.55-1354023701344,host:10.121.31.55,port:9093,
> id:3,creatorId:10.121.31.55-1354023701345,host:10.121.31.55,port:9094),0)
> [2012-11-27 16:03:36,005] INFO Terminate ZkClient event thread.
> (org.I0Itec.zkclient.ZkEventThread)
>
> My partitioner logic is doing a simple modulo on the # of partitions
> passed:
>
>     return (int) (organizationId % a_numPartitions);
>
> Did I miss a step setting up the topics?
>
> Thanks,
>
> Chris
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message