kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Neha Narkhede (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-305) SyncProducer does not correctly timeout
Date Fri, 23 Mar 2012 18:59:28 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236942#comment-13236942
] 

Neha Narkhede commented on KAFKA-305:
-------------------------------------

v2 looks good. 

Regarding the test failure, I debugged it and see a probable bug with either Zookeeper or
ZkClient. See below - 

[info] Test Starting: testZKSendWithDeadBroker(kafka.producer.ProducerTest)
Shutting down broker 0
[2012-03-23 11:50:36,870] DEBUG Deleting ephemeral node /brokers/ids/0 for session 0x13640e55f240013
(org.apache.zookeeper.server.DataTree:831)
[2012-03-23 11:50:36,873] DEBUG Deleting ephemeral node /brokers/topics/new-topic/partitions/3/leader
for session 0x13640e55f240013 (org.apache.zookeeper.server.DataTree:831)
[2012-03-23 11:50:36,873] DEBUG Deleting ephemeral node /brokers/topics/new-topic/partitions/1/leader
for session 0x13640e55f240013 (org.apache.zookeeper.server.DataTree:831)
[2012-03-23 11:50:36,873] DEBUG Deleting ephemeral node /brokers/topics/new-topic/partitions/2/leader
for session 0x13640e55f240013 (org.apache.zookeeper.server.DataTree:831)
[2012-03-23 11:50:36,873] DEBUG Deleting ephemeral node /brokers/topics/new-topic/partitions/0/leader
for session 0x13640e55f240013 (org.apache.zookeeper.server.DataTree:831)
Shut down broker 0
Restarting broker 0
[2012-03-23 11:50:45,194] DEBUG Deleting ephemeral node /brokers/ids/1 for session 0x13640e55f24001b
(org.apache.zookeeper.server.DataTree:831)
[error] Test Failed: testZKSendWithDeadBroker(kafka.producer.ProducerTest)
java.lang.RuntimeException: A broker is already registered on the path /brokers/ids/0. This
probably indicates that you either have configured a brokerid that is already in use, or else
you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears
to be re-registering.
	at kafka.utils.ZkUtils$.registerBrokerInZk(ZkUtils.scala:109)
	at kafka.server.KafkaZooKeeper.kafka$server$KafkaZooKeeper$$registerBrokerInZk(KafkaZooKeeper.scala:60)
	at kafka.server.KafkaZooKeeper.startup(KafkaZooKeeper.scala:52)
	at kafka.server.KafkaServer.startup(KafkaServer.scala:84)
	at kafka.producer.ProducerTest.testZKSendWithDeadBroker(ProducerTest.scala:173)

Notice that after shutting down broker 0, the ephemeral node was deleted from its in memory
data tree. That happens part of the close session workflow. Still, when we try to create the
ephemeral node again, it complains that it already exists. 

I'll come back to this zookeeper bug later. I'd say lets checkin this test since it helps
reproduce this zk bug. 

I think your patch looks good. 
                
> SyncProducer does not correctly timeout
> ---------------------------------------
>
>                 Key: KAFKA-305
>                 URL: https://issues.apache.org/jira/browse/KAFKA-305
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.7, 0.8
>            Reporter: Prashanth Menon
>            Priority: Critical
>         Attachments: KAFKA-305-v1.patch, KAFKA-305-v2.patch
>
>
> So it turns out that using the channel in SyncProducer like we are to perform blocking
reads will not trigger socket timeouts (though we set it) and will block forever which is
bad.  This bug identifies the issue: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4614802
and this article presents a potential work-around: http://stackoverflow.com/questions/2866557/timeout-for-socketchannel
for workaround. The work-around is a simple solution that involves creating a separate ReadableByteChannel
instance for timeout-enabled reads.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message