kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Egerton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6417) plugin.path pointing at a plugin directory causes ClassNotFoundException
Date Wed, 28 Mar 2018 21:27:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16418143#comment-16418143
] 

Chris Egerton commented on KAFKA-6417:
--------------------------------------

[~cotedm] Alright, took another look at the Connect code base and have some bad news :( It
looks like there are enough different places that the {{ClassNotFoundException}} can be
thrown from at enough different times that catching it and displaying a useful error message
would be harder than I originally thought. On top of that, it would be pretty difficult to
differentiate between "It looks like this JAR is missing because you specified an incorrect
plugin path" and "This JAR is actually missing" in most cases, since the exception will have
occurred after the initial plugin class was loaded and outside any code that deals with plugin
isolation.

 

As far as forcing all JAR files to go into their own directories goes, I have a few more
thoughts. If someone tries to start up connect, gets an error about JAR files found on the
plugin path (possibly also with the information that "All JAR files must be placed in a subdirectory
of a plugin path directory" or something along those lines), and wants to fix the issue, do
you think it's likely that the next step for them would be to then create a subdirectory and
then place all of their uber JARs inside that directory? It may seem like a reasonable assumption
that uber JARs would be loaded independently of each other, but if we don't have a meaningful
way of differentiating them from non-uber JARs in the Connect framework, we'd just end up
loading all of the plugins in that directory with the same classloader, potentially leading
to the same overlapping dependency issues that the whole {{plugin.path}} configuration is
meant to prevent.

 

> plugin.path pointing at a plugin directory causes ClassNotFoundException
> ------------------------------------------------------------------------
>
>                 Key: KAFKA-6417
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6417
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 1.0.0
>            Reporter: Dustin Cote
>            Priority: Major
>
> When using the {{plugin.path}} configuration for the Connect workers, the user is expected
to specify a list containing the following per the docs:
> {quote}
> The list should consist of top level directories that include any combination of: a)
directories immediately containing jars with plugins and their dependencies b) uber-jars with
plugins and their dependencies c) directories immediately containing the package directory
structure of classes of plugins and their dependencies 
> {quote}
> This means we would expect {{plugin.path=/usr/share/plugins}} for a structure like {{/usr/share/plugins/myplugin1}},{{/usr/share/plugins/myplugin2}},
etc. However if you specify {{plugin.path=/usr/share/plugins/myplugin1}} the resulting behavior
is that dependencies for {{myplugin1}} are not properly loaded. This causes a {{ClassNotFoundException}}
that is not intuitive to debug. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message