kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Randall Hauch (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (KAFKA-6313) Kafka Core maven dependencies are missing SLF4J API
Date Wed, 06 Dec 2017 14:51:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16280260#comment-16280260
] 

Randall Hauch edited comment on KAFKA-6313 at 12/6/17 2:50 PM:
---------------------------------------------------------------

[~ijuma], you are correct that other dependencies in my project caused the SLF4J API artifact
to be missed. It wasn't an exclusion, but it was due to including a slightly older version
3.4.6 of {{org.apache.zookeeper:zookeeper}}:

{noformat}
 +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile
 |  +- org.slf4j:slf4j-api:jar:1.6.1:compile
 |  \- io.netty:netty:jar:3.7.0.Final:compile
{noformat}

Because ZK depends *directly* on SLF4J API while Kafka depends *indirectly*, Maven's [transitive
dependency rules|https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Transitive_Dependencies]
used ZK's direct dependency to resolve the version conflict and to identify the version to
use. Reordering in my project Kafka's dependency first doesn't help, though I could force
it by manually adding the SLF4J dependency (now I'd have to manage the version) or the Kafka
client dependency (which does have a direct dependency on SLF4J).

So, you are correct that the existing dependencies should have brought in the SLF4J if there
were no other conflicts/exclusions, and that we should make Kafka's dependency direct rather
than indirect. However, because this is so difficult to fix, I'd still characterize this as
more important because of how Maven's dependency conflict resolution works.





was (Author: rhauch):
[~ijuma], you are correct that other dependencies in my project caused the SLF4J API artifact
to be missed. It wasn't an exclusion, but it was due to including a slightly older version
3.4.6 of {{org.apache.zookeeper:zookeeper}}:

{noformat}
 +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile
 |  +- org.slf4j:slf4j-api:jar:1.6.1:compile
 |  \- io.netty:netty:jar:3.7.0.Final:compile
{noformat}

Because ZK depends *directly* on SLF4J API while Kafka depends *indirectly*, Maven's [transitive
dependency rules|https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Transitive_Dependencies]
used ZK's direct dependency to identify the version conflict. Reordering in my project Kafka's
dependency first doesn't help, though I could force it by manually adding the SLF4J dependency
(now I'd have to manage the version) or the Kafka client dependency (which does have a direct
dependency on SLF4J).

So, you are correct that the existing dependencies should have brought in the SLF4J if there
were no other conflicts/exclusions, and that we should make Kafka's dependency direct rather
than indirect. However, because this is so difficult to fix, I'd still characterize this as
more important because of how Maven's dependency conflict resolution works.




> Kafka Core maven dependencies are missing SLF4J API
> ---------------------------------------------------
>
>                 Key: KAFKA-6313
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6313
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 1.1.0
>            Reporter: Randall Hauch
>            Assignee: Ismael Juma
>            Priority: Minor
>             Fix For: 1.1.0
>
>
> In an application that depends on the Kafka server artifacts with:
> {code:xml}
> 	<dependency>
> 		<groupId>org.apache.kafka</groupId>
> 		<artifactId>kafka_2.11</artifactId>
> 		<version>1.1.0-SNAPSHOT</version>
> 	</dependency>
> {code}
> and then running the server programmatically, the following error occurs:
> {noformat}
> [2017-11-23 01:01:45,029] INFO Shutting down producer (kafka.producer.Producer:63)
> [2017-11-23 01:01:45,051] INFO Closing all sync producers (kafka.producer.ProducerPool:63)
> [2017-11-23 01:01:45,052] INFO Producer shutdown completed in 23 ms (kafka.producer.Producer:63)
> [2017-11-23 01:01:45,052] INFO [KafkaServer id=1] shutting down (kafka.server.KafkaServer:63)
> [2017-11-23 01:01:45,057] ERROR [KafkaServer id=1] Fatal error during KafkaServer shutdown.
(kafka.server.KafkaServer:161)
> java.lang.NoClassDefFoundError: org/slf4j/event/Level
> 	at kafka.utils.CoreUtils$.swallow$default$3(CoreUtils.scala:83)
> 	at kafka.server.KafkaServer.shutdown(KafkaServer.scala:520)
>        ...
> Caused by: java.lang.ClassNotFoundException: org.slf4j.event.Level
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:359)
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:348)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:347)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> 	... 25 more
> {noformat}
> It appears that KAFKA-1044 and [this PR|https://github.com/apache/kafka/pull/3477] removed
the use of Log4J from Core but [added use of|https://github.com/confluentinc/kafka/commit/ed8b0315a6c3705b2a163ce3ab4723234779264f#diff-52505b9374ea885e44bcb73cbc4714d6R34]
the {{org.slf4j.event.Level}} in {{CoreUtils.scala}}. The {{org.slf4j.event.Level}} class
is in the {{org.slf4j:slf4j-api}} artifact, which is currently not included in the dependencies
of {{org.apache.kafka:kafka_2.11:1.1.0-SNAPSHOT}}. Because this is needed by the server, the
SLF4J API library probably needs to be added to the dependencies.
> [~viktorsomogyi] and [~ijuma], was this intentional, or is it intended that the SLF4J
API be marked as {{provided}}? BTW, I marked this as CRITICAL just because this probably needs
to be sorted out before the release.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message