kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Gustafson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4238) consumer-subscription not working, when accessing a newly created topic immediately after its creation with the AdminUtils
Date Fri, 30 Sep 2016 21:05:20 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537069#comment-15537069
] 

Jason Gustafson commented on KAFKA-4238:
----------------------------------------

[~apache@flwi.de] The consumer periodically refetches topic metadata internally according
to the configuration {{metadata.max.age.ms}}. If a topic is just created, you may have to
wait up to this amount of time for the consumer to detect it. If you want to detect the topic
sooner, adjust this setting lower.

> consumer-subscription not working, when accessing a newly created topic immediately after
its creation with the AdminUtils
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-4238
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4238
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.10.0.0, 0.10.0.1
>            Reporter: Florian Witteler
>
> I created a test-project to reproduce the bug.
> https://github.com/FloWi/kafka-topic-creation-bug
> We use a docker container that creates a fresh topic before a testsuite gets executed
(see {{trait FreshKafkaTopics}}). That trait uses the AdminUtils to create the topic. 
> If we access the newly created topic directly after its creation, the subscriber is broken.
It sometimes works though (<5%), so it seems to be a race-condition.
> If I put a {{Thread.sleep(1000)}} after the topic-creation, everything's fine though.
> So, the problem is twofold:
> - {{AdminUtils.createTopic}} should block until the topic-creation is completed
> - {{new KafkaConsumer[String, String](props).subscribe(util.Arrays.asList(topic))}} should
throw an exception, when the topic is "not ready"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message