kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manikumar Reddy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4164) Kafka produces excessive logs when publishing message to non-existent topic
Date Fri, 16 Sep 2016 05:44:20 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15495450#comment-15495450
] 

Manikumar Reddy commented on KAFKA-4164:
----------------------------------------

This is a valid concern. Currently metadata TimeoutException does't include underlying cause
(UNKNOWN_TOPIC_OR_PARTITION, LEADER_NOT_AVAILABLE) etc. 
It may be tricky to add this, since wait and metadata fetch are running in different threads.

> Kafka produces excessive logs when publishing message to non-existent topic
> ---------------------------------------------------------------------------
>
>                 Key: KAFKA-4164
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4164
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Vimal Sharma
>
> When a message is published to a topic which is not already created(and auto.create.topics.enable
is set to false), Kafka produces excessive WARN logs stating that metadata could not be fetched.
Below are the logs
> 2016-08-22 06:43:47,655 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1177 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:47,756 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1178 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:47,858 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1179 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:47,961 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1180 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,062 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1181 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,165 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1182 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,265 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1183 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,366 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1184 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,467 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1185 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> The error is not communicated to the caller so if these logs are suppressed by setting
Kafka log level to ERROR, there is no way to debug the issue. It would be helpful if the error
message( for example {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}) can be communicated to the caller.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message