kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jun Rao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-899) LeaderNotAvailableException the first time a new message for a partition is processed.
Date Wed, 01 Oct 2014 23:02:34 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155742#comment-14155742
] 

Jun Rao commented on KAFKA-899:
-------------------------------

We started doing that classification in the new java producer. For example, there are certain
exceptions are of RetriableException. Transient failures like leader not available are in
that category. Exceptions like MessageTooLarge are in a different category. Perhaps you can
take a look at that in the new producer and see if that makes sense.

> LeaderNotAvailableException the first time a new message for a partition is processed.
> --------------------------------------------------------------------------------------
>
>                 Key: KAFKA-899
>                 URL: https://issues.apache.org/jira/browse/KAFKA-899
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.8.0
>            Reporter: Jason Rosenberg
>            Assignee: Jun Rao
>             Fix For: 0.8.2
>
>         Attachments: kafka-899.patch, kafka-899_v2.patch, kafka-899_v3.patch
>
>
> I'm porting some unit tests from 0.7.2 to 0.8.0.  The test does the following, all embedded
in the same java process:
> -- spins up a zk instance
> -- spins up a kafka server using a fresh log directory
> -- creates a producer and sends a message
> -- creates a high-level consumer and verifies that it can consume the message
> -- shuts down the consumer
> -- stops the kafka server
> -- stops zk
> The test seems to be working fine now, however, I consistently see the following exceptions
(which from poking around the mailing list seem to be expected?).  If these are expected,
can we suppress the logging of these exceptions, since it clutters the output of tests, and
presumably, clutters the logs of the running server/consumers, during clean startup and shutdown......
> When I call producer.send(), I get:
> 1071 [main] WARN kafka.producer.BrokerPartitionInfo  - Error while fetching metadata
	partition 0	leader: none	replicas: 	isr: 	isUnderReplicated: false for topic partition [test-topic,0]:
[class kafka.common.LeaderNotAvailableException]
> 1081 [main] WARN kafka.producer.async.DefaultEventHandler  - Failed to collate messages
by topic,partition due to
> kafka.common.LeaderNotAvailableException: No leader for any partition
> 	at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartition(DefaultEventHandler.scala:212)
> 	at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:150)
> 	at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:148)
> 	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
> 	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
> 	at kafka.producer.async.DefaultEventHandler.partitionAndCollate(DefaultEventHandler.scala:148)
> 	at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:94)
> 	at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:72)
> 	at kafka.producer.Producer.send(Producer.scala:74)
> 	at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> 	at com.squareup.kafka.server.KafkaServerTest.produceAndConsumeMessage(KafkaServerTest.java:98)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:69)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:48)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:292)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> 	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
> 	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
> 	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> 1133 [kafka-request-handler-1] WARN kafka.server.HighwaterMarkCheckpoint  - No highwatermark
file is found. Returning 0 as the highwatermark for partition [test-topic,0]
> 	...
>         ...
> It would be great if instead of this exception, it would just log a meaningful message,
like:
> "No leader was available for partition X, one will now be created"
> Jason



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message