atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vimal Sharma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-1131) Atlas hooks produce excess Kafka logs if Kafka topic is not created already
Date Mon, 22 Aug 2016 11:15:22 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15430574#comment-15430574
] 

Vimal Sharma commented on ATLAS-1131:
-------------------------------------

One solution is to set Kafka log level to ERROR in individual hooks. For example, in hive-log4j.xml,
setting
log4j.logger.org.apache.kafka=ERROR
suppresses these logs. But setting this property would make it difficult to debug other Kafka
topic related issues. 

> Atlas hooks produce excess Kafka logs if Kafka topic is not created already
> ---------------------------------------------------------------------------
>
>                 Key: ATLAS-1131
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1131
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Vimal Sharma
>            Assignee: Vimal Sharma
>
> Hooks for Atlas publish messages to a Kafka topic named ATLAS_HOOK. If this topic is
not present and Atlas does not have permission to create this topic, Kafka produces excessive
logs as below (example in /tmp/hive/hive.log).
> 2016-08-22 06:43:47,655 WARN  [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1177 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:47,756 WARN  [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1178 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:47,858 WARN  [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1179 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:47,961 WARN  [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1180 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,062 WARN  [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1181 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,165 WARN  [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1182 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,265 WARN  [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1183 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,366 WARN  [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1184 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,467 WARN  [kafka-producer-network-thread | producer-1]: clients.NetworkClient
(NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation
id 1185 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message